Cancer auxiliary detection system and method based on triple network

Abstract

The invention discloses a cancer auxiliary detection system and method based on a triple network, and the method comprises the steps: taking the minimization of a loss function of a triple network model as a target; performing iterative training on the triple network model by utilizing the anchor point sample pathological image subset, the positive sample pathological image subset and the negative sample pathological image subset until the loss value of the loss function is smaller than or equal to a preset loss value threshold value to obtain a pathological image recognition model; inputting an acquired to-be-recognized pathological image into the pathological image recognition model to obtain a pathological area graph containing at least one pathological feature; and inputting the pathological area graph containing the at least one pathological feature into a preset pathological type discrimination model to obtain a pathological type prediction result corresponding to the to-be-identified pathological image. Accurate cancer classification prediction can be carried out according to the feature vectors of the pathological sections, and the cancer detection accuracy and reliability of intelligent pathology are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of pathological image recognition, and in particular relates to a cancer auxiliary detection system and method based on a triplet network. Background Art

[0002] Pathological images play a vital role in medical diagnosis. At present, the digital processing of pathological images has become a hot research direction in the medical field. Digital pathological image technology can further improve the efficiency and accuracy of doctors' diagnosis. And pathological image technology gradually involves multiple fields such as computer vision, image processing and artificial intelligence.

[0003] In the field of intelligent pathology, pathology images are important tools for doctors to diagnose and study diseases. However, the staining diversity of pathology sections has an impact on cancer classification tasks. Specifically, the staining diversity of pathology sections will lead to poor generalization performance of the model, making it impossible to make accurate cancer classification predictions based on the feature vectors of pathology sections. Summary of the invention

[0004] The present invention provides a cancer auxiliary detection system and method based on a triplet network, which are used to solve the technical problem that accurate cancer classification prediction cannot be performed based on the feature vectors of pathological sections.

[0005] In a first aspect, the present invention provides a cancer auxiliary detection system based on a triplet network, comprising:

[0006] An acquisition module is configured to acquire a sample pathology image set, wherein the sample pathology image set includes an anchor point sample pathology image subset, a positive sample pathology image subset, and a negative sample pathology image subset;

[0007] A training module is configured to iteratively train the triplet network model with the goal of minimizing the loss function of the triplet network model by using the anchor sample pathological image subset, the positive sample pathological image subset and the negative sample pathological image subset until the loss value of the loss function is less than or equal to a preset loss value threshold, or until the number of iterations of the triplet network model is greater than or equal to a number threshold, to obtain a pathological image recognition model;

[0008] A recognition module, configured to input the acquired pathological image to be recognized into the pathological image recognition model, and the pathological image recognition model outputs a pathological area map containing at least one pathological feature;

[0009] The discrimination module is configured to input a pathological region image containing at least one pathological feature into a preset pathological type discrimination model, and the pathological type discrimination model outputs a pathological type prediction result corresponding to the pathological image to be identified.

[0010] Further, the construction module includes a cropping unit for cropping the sample pathology image to obtain a cropped sample pathology image;

[0011] The cutting unit comprises:

[0012] Get subunits for:

[0013] Obtaining the pixel value of each pixel in the sample pathological image;

[0014] Determine a gradient value for each pixel based on the pixel value of each pixel;

[0015] Compare the gradient value of each pixel with a preset gradient threshold, determine the pixel whose gradient value is greater than the preset gradient threshold as a first edge pixel, and obtain a first edge pixel set; determine a first edge image based on the first edge pixel set;

[0016] Convert the first edge image into grayscale to obtain a grayscale image;

[0017] Get the gray value of each pixel in the gray image;

[0018] Generate a grayscale histogram based on the grayscale value of each pixel in the grayscale image;

[0019] Determine the grayscale value range on both sides of the main peak of the area where the main peak is located according to the grayscale histogram, and determine the density value corresponding to each grayscale value in the grayscale value range on both sides of the main peak based on the grayscale value range on both sides of the main peak and the number of pixels corresponding to each grayscale value in the grayscale value range on both sides of the main peak;

[0020] Calculate the Euclidean distance between pixels corresponding to two adjacent density values;

[0021] Calculate the discreteness of each pixel based on the Euclidean distance between two adjacent density values ​​and the corresponding pixel points; determine the pixel points whose discreteness is less than a preset discreteness threshold as discrete points;

[0022] Removing discrete points from the first edge image to obtain a second edge image;

[0023] Performing image matching on the second edge image and the sample pathology image, and superimposing the second edge image and the sample pathology image according to the matching result;

[0024] The cropping subunit is used to perform cropping according to the second image to obtain a cropped sample pathology image.

[0025] Furthermore, the system further comprises:

[0026] The construction module is configured to preprocess each sample pathology image, and construct a sample pathology image set based on the type of each preprocessed sample pathology image.

[0027] Furthermore, the preprocessing includes cropping of the sample pathology image, scaling of the sample pathology image, and color standardization of the sample pathology image.

[0028] Furthermore, the calculation process of the loss value of the loss function includes:

[0029] Calculating a first distance corresponding to the triplet network model according to the anchor sample image features of the anchor sample pathological image and the positive sample image features of the positive sample pathological image, wherein the first distance represents a distance between a reference image feature and a positive sample image feature in the triplet network model;

[0030] Calculating a second distance corresponding to the triplet network model according to the anchor sample image features of the anchor sample pathological image and the negative sample image features of the negative sample pathological image, wherein the second distance represents a distance between a reference image feature and a negative sample image feature in the triplet network model;

[0031] A loss value of the loss function is determined according to the first distance and the second distance.

[0032] Furthermore, the identification module includes:

[0033] A preprocessing unit, configured to preprocess the acquired pathological image to be identified to obtain a target pathological image to be identified, wherein the preprocessing includes cropping the sample pathological image, adjusting the scale of the sample pathological image, and color standardization of the sample pathological image;

[0034] An extraction unit, configured to extract a feature region of the target pathological image to be identified based on a feature extractor, to obtain a feature region map corresponding to the target pathological image to be identified;

[0035] The output unit is configured to input the feature area map corresponding to the target pathological image to be identified into the pathological image recognition model, and the pathological image recognition model outputs a pathological area map containing at least one pathological feature.

[0036] Furthermore, the feature area map only includes at least one feature area.

[0037] Furthermore, the feature extractor is a single-step multi-frame target detection model.

[0038] Furthermore, the system further comprises:

[0039] an acquisition module, configured to acquire the at least one pathological feature and annotation data corresponding to the at least one pathological feature;

[0040] The iteration module is configured to train the neural network model according to the at least one pathological feature and the labeled data corresponding to the at least one pathological feature until the number of iterations of the neural network model is greater than or equal to a number threshold, thereby obtaining a pathological type discrimination model.

[0041] Furthermore, the system further comprises: an enhancement module, which is used to perform enhancement processing on the pathological image to be identified before the recognition module inputs the acquired pathological image to be identified into the pathological image recognition model;

[0042] The enhancement module comprises:

[0043] Computing unit for:

[0044] Perform grayscale conversion on the pathological image to be identified to obtain a grayscale image;

[0045] Randomly select a pixel point in the grayscale image as a first pixel point; determine a first image to be processed with the first pixel point as the center and a first preset distance as the radius;

[0046] Obtaining the grayscale value of each pixel in the first image to be processed and calculating the grayscale mean to obtain a first grayscale mean;

[0047] Obtaining the median value of each pixel in the first image to be processed;

[0048] Calculate the absolute value of the difference between the first grayscale mean and the median to obtain a difference parameter;

[0049] Compare the difference parameter with a preset difference threshold, and when it is determined that the difference parameter is greater than or equal to the preset difference threshold, obtain the smaller value between the first grayscale mean and the median value to obtain a screening parameter;

[0050] Determine the pixel points whose grayscale values ​​are less than the screening parameter in the first image area to be processed, and generate a second image to be processed;

[0051] Randomly select a pixel point in the second image to be processed as the second pixel point; determine the third image to be processed with the second pixel point as the center and the second preset distance as the radius; the second preset distance is smaller than the first preset distance;

[0052] Obtaining the grayscale value of each pixel in the third image to be processed and calculating the grayscale mean to obtain a second grayscale mean;

[0053] Determine the maximum grayscale value and the minimum grayscale value of the pixels in the third image to be processed, calculate the difference between the maximum grayscale value and the minimum grayscale value, and obtain the grayscale difference;

[0054] Determine the enhancement multiple of the second pixel based on a preset algorithm based on the gray value of the second pixel, the second gray mean and the gray difference;

[0055] Traversing each pixel in the third image to be processed, and determining an enhancement multiple for each pixel in the third image to be processed;

[0056] Based on the enhancement multiple of each pixel in the third image to be processed, the third image to be processed is enhanced to obtain an enhanced pathological image to be identified;

[0057] The output unit is used to input the enhanced pathological image to be identified into the pathological image recognition model.

[0058] Furthermore, the identification module includes:

[0059] A grayscale processing unit, configured to perform grayscale processing on the pathological region image containing at least one pathological feature, and perform binarization processing on the grayscale processed pathological image by using a maximum inter-class variance method to obtain a binary image;

[0060] a denoising unit, configured to denoise the binary image by a neighborhood average algorithm to generate a target pathological area map containing at least one pathological feature;

[0061] The discrimination unit is configured to input a target pathological area image containing at least one pathological feature into a preset pathological type discrimination model, and the pathological type discrimination model outputs a pathological type prediction result corresponding to the pathological image to be identified.

[0062] In a second aspect, the present invention provides a cancer auxiliary detection method based on a triplet network, comprising:

[0063] Acquire a sample pathology image set, wherein the sample pathology image set includes an anchor sample pathology image subset, a positive sample pathology image subset, and a negative sample pathology image subset;

[0064] Taking minimization of the loss function of the triple network model as the goal, the triple network model is iteratively trained using the anchor sample pathological image subset, the positive sample pathological image subset and the negative sample pathological image subset until the loss value of the loss function is less than or equal to a preset loss value threshold, or until the number of iterations of the triple network model is greater than or equal to a number threshold, to obtain a pathological image recognition model;

[0065] Inputting the acquired pathological image to be identified into the pathological image recognition model, and the pathological image recognition model outputs a pathological area map containing at least one pathological feature;

[0066] A pathological region image including at least one pathological feature is input into a preset pathological type discrimination model, and the pathological type discrimination model outputs a pathological type prediction result corresponding to the pathological image to be identified.

[0067] In a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the cancer auxiliary detection method based on a triplet network of any embodiment of the present invention.

[0068] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the program instructions are executed by a processor, the processor executes the steps of the cancer auxiliary detection method based on a triplet network of any embodiment of the present invention.

[0069] The cancer auxiliary detection method and system based on the triplet network of the present application aims to minimize the loss function of the triplet network model, and uses the anchor sample pathological image subset, the positive sample pathological image subset and the negative sample pathological image subset to iteratively train the triplet network model until the loss value of the loss function is less than or equal to the preset loss value threshold. The acquired pathological image to be identified is then input into the pathological image recognition model, and the pathological image recognition model outputs a pathological area map containing at least one pathological feature, and the pathological area map containing at least one pathological feature is input into a preset pathological type discrimination model, and the pathological type discrimination model outputs a pathological type prediction result corresponding to the pathological image to be identified, which can accurately predict cancer classification based on the feature vector of the pathological slice, thereby improving the accuracy and reliability of cancer detection in intelligent pathology. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0071] Figure 1 A flowchart of a cancer auxiliary detection method based on a triplet network provided in one embodiment of the present invention;

[0072] Figure 2 A structural block diagram of a cancer auxiliary detection system based on a triplet network provided in one embodiment of the present invention;

[0073] Figure 3It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] The cancer auxiliary detection system and method based on the triplet network of the present application are for non-diagnostic purposes, that is, they are not used for direct diagnosis, or they are only used to assist the doctor's diagnosis. That is, the scheme involved in the system and method will not directly obtain the final diagnostic result, but only obtain reference data that is helpful for the doctor's diagnosis.

[0076] See also Figure 1 , which shows a flow chart of a cancer auxiliary detection method based on a triplet network of the present application.

[0077] like Figure 1 As shown, the cancer auxiliary detection method based on the triplet network specifically includes the following steps:

[0078] Step S101 : obtaining a sample pathology image set, wherein the sample pathology image set includes an anchor point sample pathology image subset, a positive sample pathology image subset, and a negative sample pathology image subset.

[0079] The anchor samples, negative samples and positive samples in the anchor sample pathology image subset, the positive sample pathology image subset and the negative sample pathology image subset constitute a triplet.

[0080] The anchor samples in the anchor sample pathology image subset can also be called Anchor in the triple. The negative sample pathology image subset and the positive sample pathology image subset are sample sets constructed to form a triple with the anchor sample pathology image subset. There is a corresponding relationship between the positive sample, the negative sample and the anchor sample.

[0081] It should be noted that each sample pathology image is preprocessed, and then each preprocessed sample pathology image is used to construct a sample pathology image set based on type. The preprocessing includes cropping the sample pathology image, adjusting the scale of the sample pathology image, and color standardization of the sample pathology image.

[0082] Step S102, with the goal of minimizing the loss function of the triplet network model, the triplet network model is iteratively trained using the anchor sample pathological image subset, the positive sample pathological image subset and the negative sample pathological image subset until the loss value of the loss function is less than or equal to a preset loss value threshold, or until the number of iterations of the triplet network model is greater than or equal to a number threshold, to obtain a pathological image recognition model.

[0083] In this step, according to the anchor sample image features of the anchor sample pathological image and the positive sample image features of the positive sample pathological image, the first distance corresponding to the triplet network model is calculated, and the first distance represents the distance between the reference image features and the positive sample image features in the triplet network model; according to the anchor sample image features of the anchor sample pathological image and the negative sample image features of the negative sample pathological image, the second distance corresponding to the triplet network model is calculated, and the second distance represents the distance between the reference image features and the negative sample image features in the triplet network model; the loss value of the loss function is determined according to the first distance and the second distance.

[0084] Step S103: input the acquired pathological image to be identified into the pathological image recognition model, and the pathological image recognition model outputs a pathological area map containing at least one pathological feature.

[0085] In this step, the acquired pathological image to be identified is preprocessed to obtain a target pathological image to be identified, wherein the preprocessing includes cropping the sample pathological image, adjusting the scale of the sample pathological image, and color standardization of the sample pathological image;

[0086] Extracting feature regions of the target pathological image to be identified based on the feature extractor to obtain a feature region map corresponding to the target pathological image to be identified, wherein the feature region map only includes at least one feature region;

[0087] The feature area map corresponding to the target pathological image to be identified is input into the pathological image recognition model, and the pathological image recognition model outputs a pathological area map containing at least one pathological feature.

[0088] It should be noted that the feature extractor is a single-step multi-frame target detection model. It is understandable that the single-step multi-frame target detection model is a target detection model based on deep learning, which can be used for feature extraction and lesion area detection of pathological images.

[0089] Step S104, inputting the pathological region image containing at least one pathological feature into a preset pathological type discrimination model, and the pathological type discrimination model outputs a pathological type prediction result corresponding to the pathological image to be identified.

[0090] Acquire at least one pathological feature and labeled data corresponding to the at least one pathological feature; train a neural network model according to the at least one pathological feature and the labeled data corresponding to the at least one pathological feature until the number of iterations of the neural network model is greater than or equal to a threshold number, thereby obtaining a pathological type discrimination model.

[0091] The pathological area map containing at least one pathological feature is gray-processed, and the gray-processed pathological image is binarized by the maximum inter-class variance method to obtain a binary image; the binary image is denoised by the neighborhood average algorithm to generate a target pathological area map containing at least one pathological feature; the target pathological area map containing at least one pathological feature is input into a preset pathological type discrimination model, and the pathological type discrimination model outputs a pathological type prediction result corresponding to the pathological image to be identified.

[0092] It should be noted that the pathological region image containing at least one pathological feature is gray-scaled, and the gray-scaled pathological image is binarized by the maximum inter-class variance method to obtain a binary image:

[0093] The base color of a pathological area map containing at least one pathological feature is adjusted according to a weighted average method to achieve grayscale processing; the number of pixels corresponding to each grayscale level in the pathological area map containing at least one pathological feature after grayscale processing is detected to obtain a grayscale mapping table; the grayscale mapping table is traversed based on a maximum inter-class variance method to generate a target grayscale threshold; the pathological area map containing at least one pathological feature after grayscale processing is binarized according to the target grayscale threshold to obtain a binary image.

[0094] Specifically, the grayscale mapping table is traversed based on the maximum inter-class variance method to obtain the grayscale traversal result; the image processing type is determined according to the pathological information corresponding to the pathological area map containing at least one pathological feature; the corresponding operation of the image processing type is confirmed through the image processing relationship table; when the image processing type corresponds to the edge feature enhancement operation, the grayscale threshold of the image is adjusted through the Matlab platform combined with the grayscale traversal result to complete the edge feature enhancement operation and obtain the current grayscale threshold; the current grayscale threshold is used as the target grayscale threshold.

[0095] In summary, the method of the present application aims to minimize the loss function of the triplet network model, and uses the anchor sample pathology image subset, the positive sample pathology image subset and the negative sample pathology image subset to iteratively train the triplet network model until the loss value of the loss function is less than or equal to the preset loss value threshold, and then inputs the acquired pathology image to be identified into the pathology image recognition model, and the pathology image recognition model outputs a pathology area map containing at least one pathology feature, and the pathology area map containing at least one pathology feature is input into the preset pathology type discrimination model, and the pathology type discrimination model outputs a pathology type prediction result corresponding to the pathology image to be identified, which can perform accurate cancer classification prediction based on the feature vector of the pathology section, thereby improving the accuracy and reliability of cancer detection of intelligent pathology.

[0096] The construction module includes a cropping unit, which is used to crop the sample pathology image to obtain a cropped sample pathology image;

[0097] The cutting unit comprises:

[0098] Get subunits for:

[0099] Obtaining the pixel value of each pixel in the sample pathological image;

[0100] Determine a gradient value for each pixel based on the pixel value of each pixel;

[0101] Compare the gradient value of each pixel with a preset gradient threshold, determine the pixel whose gradient value is greater than the preset gradient threshold as a first edge pixel, and obtain a first edge pixel set; determine a first edge image based on the first edge pixel set;

[0102] Convert the first edge image into grayscale to obtain a grayscale image;

[0103] Get the gray value of each pixel in the gray image;

[0104] Generate a grayscale histogram based on the grayscale value of each pixel in the grayscale image;

[0105] Determine the grayscale value range on both sides of the main peak of the area where the main peak is located according to the grayscale histogram, and determine the density value corresponding to each grayscale value in the grayscale value range on both sides of the main peak based on the grayscale value range on both sides of the main peak and the number of pixels corresponding to each grayscale value in the grayscale value range on both sides of the main peak;

[0106] Calculate the Euclidean distance between pixels corresponding to two adjacent density values;

[0107] Calculate the discreteness of each pixel based on the Euclidean distance between two adjacent density values ​​and the corresponding pixel points; determine the pixel points whose discreteness is less than a preset discreteness threshold as discrete points;

[0108] Removing discrete points from the first edge image to obtain a second edge image;

[0109] Performing image matching on the second edge image and the sample pathology image, and superimposing the second edge image and the sample pathology image according to the matching result;

[0110] The cropping subunit is used to perform cropping according to the second image to obtain a cropped sample pathology image.

[0111] In this embodiment, the specific implementation method of determining the gradient value of each pixel point based on the pixel value of each pixel point is as follows: take any pixel point, calculate the difference between the pixel value of the pixel point to the right of the pixel point and the pixel value of the pixel point, as the horizontal gradient value of the pixel point; calculate the difference between the pixel value of the pixel point above the pixel point and the pixel value of the pixel point, as the vertical gradient value of the pixel point; obtain the gradient value of the pixel point based on the square root of the sum of the squares of the horizontal gradient value and the vertical gradient value; traverse each pixel point to obtain the gradient value of each pixel point.

[0112] In this embodiment, the larger the gradient value of a pixel point is, the greater the difference between the pixel point and surrounding pixels is; this difference can be understood as the pixel point being part of the edge of the image, because the edge usually corresponds to a position where the pixel value changes rapidly.

[0113] In this embodiment, the gradient value of each pixel point is compared with a preset gradient threshold, edge pixels with relatively large gradient values ​​are screened out, and a first edge image is determined based on the edge pixels.

[0114] In this embodiment, the specific implementation method of generating a grayscale histogram based on the grayscale value of each pixel in the grayscale image is as follows: obtaining the grayscale value of each pixel in the grayscale image and the total number of pixels, sorting all the pixels of the image according to the grayscale value, counting the number of occurrences or frequencies of each grayscale value, and drawing a bar graph with the grayscale value as the horizontal axis and the frequency of occurrence as the vertical axis; the grayscale histogram is a statistical tool used to describe the grayscale distribution of an image in digital image processing. It counts the grayscale value of each pixel in the image and its frequency of occurrence to reflect the number of pixels of a certain grayscale level in the image. The grayscale histogram is mainly used to analyze and understand the brightness distribution of an image, which can help us understand the basic features of the image, such as brightness and contrast. It is usually used in the fields of image enhancement, image compression, pattern recognition, etc. In pattern recognition, the grayscale histogram can provide information such as texture and structure of the image for feature extraction and classification.

[0115] In this embodiment, the grayscale value range on both sides of the main peak of the area where the main peak is located is determined according to the grayscale histogram, and based on the grayscale value range on both sides of the main peak and the number of pixels corresponding to each grayscale value in the grayscale value range on both sides of the main peak, the density value corresponding to each grayscale value in the grayscale value range on both sides of the main peak is determined. The specific implementation method is: the density value corresponding to each grayscale value is determined according to the ratio of the number of pixels corresponding to each grayscale value in the grayscale value range on both sides of the main peak to the total number of pixels in the grayscale value range on both sides of the main peak.

[0116] In this embodiment, the specific implementation method for determining the area where the main peak is located is: performing least squares fitting on the grayscale histogram to obtain a fitting curve of the grayscale histogram, obtaining all minimum points in the fitting curve, calculating the total number of pixels in the grayscale interval where two adjacent minimum values ​​are located, and determining the maximum value of the total number of pixels as the main peak.

[0117] In this embodiment, the specific formula for calculating the dispersion is:

[0118]

[0119] Among them, T i is the discreteness of pixel i, A i is the Euclidean distance from the pixel to the pixel i in the gray value corresponding to the density value adjacent to the gray value where the pixel i is located, B i is the density value of pixel i, a is the preset adjustment parameter, a∈(0,1).

[0120] The above technical solution has the following beneficial effects: obtaining the pixel value of each pixel in the sample pathological image, determining the gradient value of each pixel based on the pixel value of each pixel, comparing the gradient value of each pixel with a preset gradient threshold, determining the pixel whose gradient value is greater than the preset gradient threshold as the first edge pixel, obtaining the first edge pixel set, and screening out the edge pixel set by calculating the gradient value. The larger the gradient value of the pixel, the greater the difference between the pixel and the surrounding pixels. This difference can be understood as the pixel being part of the edge of the image, because the edge usually corresponds to a position where the pixel value changes rapidly. The first edge image is gray-converted to obtain a gray-scale image. The gray-converted first edge image can reduce the dimension of the image data and simplify the image processing steps while retaining sufficient information for subsequent processing. A gray-scale histogram is generated based on the gray-scale value of each pixel in the gray-scale image. The histogram can provide information such as texture and structure of the image, which is used for feature extraction and classification; the grayscale value range on both sides of the main peak in the area where the main peak is located is determined according to the grayscale histogram. The grayscale value range where the main peak is located is an area where the grayscale values ​​of edge pixels are relatively concentrated. The grayscale value range within the grayscale value range where the main peak is located is selected, and the grayscale value range outside the grayscale value range where the peak is located is deleted to complete the first screening of edge pixels; the density value corresponding to each grayscale value in the grayscale value range on both sides of the main peak is calculated and determined, and the Euclidean distance between the pixels corresponding to two adjacent density values ​​is calculated. The discreteness of each pixel is calculated based on the Euclidean distance between the two adjacent density values ​​and the corresponding pixels. The pixel points whose discreteness is less than the preset discreteness threshold are deleted to complete the second screening of edge pixels; through the two screenings of edge pixels, the accuracy of edge pixel screening is improved, providing an accurate basis for the subsequent second image cropping.

[0121] The system further comprises: an enhancement module, which is used to perform enhancement processing on the pathological image to be identified before the recognition module inputs the acquired pathological image to be identified into the pathological image recognition model;

[0122] The enhancement module comprises:

[0123] Computing unit for:

[0124] Perform grayscale conversion on the pathological image to be identified to obtain a grayscale image;

[0125] Randomly select a pixel point in the grayscale image as a first pixel point; determine a first image to be processed with the first pixel point as the center and a first preset distance as the radius;

[0126] Obtaining the grayscale value of each pixel in the first image to be processed and calculating the grayscale mean to obtain a first grayscale mean;

[0127] Obtaining the median value of each pixel in the first image to be processed;

[0128] Calculate the absolute value of the difference between the first grayscale mean and the median to obtain a difference parameter;

[0129] Compare the difference parameter with a preset difference threshold, and when it is determined that the difference parameter is greater than or equal to the preset difference threshold, obtain the smaller value between the first grayscale mean and the median value to obtain a screening parameter;

[0130] Determine the pixel points whose grayscale values ​​are less than the screening parameter in the first image area to be processed, and generate a second image to be processed;

[0131] Randomly select a pixel point in the second image to be processed as the second pixel point; determine the third image to be processed with the second pixel point as the center and the second preset distance as the radius; the second preset distance is smaller than the first preset distance;

[0132] Obtaining the grayscale value of each pixel in the third image to be processed and calculating the grayscale mean to obtain a second grayscale mean;

[0133] Determine the maximum grayscale value and the minimum grayscale value of the pixels in the third image to be processed, calculate the difference between the maximum grayscale value and the minimum grayscale value, and obtain the grayscale difference;

[0134] Determine the enhancement multiple of the second pixel based on a preset algorithm based on the gray value of the second pixel, the second gray mean and the gray difference;

[0135] Traversing each pixel in the third image to be processed, and determining an enhancement multiple for each pixel in the third image to be processed;

[0136] Based on the enhancement multiple of each pixel in the third image to be processed, the third image to be processed is enhanced to obtain an enhanced pathological image to be identified;

[0137] The output unit is used to input the enhanced pathological image to be identified into the pathological image recognition model.

[0138] In this embodiment, the difference parameter is the absolute value of the difference between the first grayscale mean and the median;

[0139] In this embodiment, the screening parameter is the smaller value between the first grayscale mean value and the median value;

[0140] In this embodiment, the pixels whose grayscale values ​​are less than the screening parameters in the first image to be processed are determined, and the second image to be processed is generated, in order to screen out the pixels with relatively low brightness values ​​in the current first image to be processed as the pixels to be subsequently enhanced.

[0141] In this embodiment, the preset algorithm is:

[0142]

[0143] Among them, P i is the enhancement multiple of pixel i, Q is the second grayscale mean, M is the grayscale difference, N i is the gray value of pixel i, and n is the total number of pixels in the third image to be processed.

[0144] The algorithm of the present application is used to calculate the enhancement multiple of the pixel point. The enhancement multiple of the pixel point is determined from multiple dimensions through the grayscale value of the second pixel point, the second grayscale mean, and the grayscale difference, thereby making the image enhancement effect more obvious and improving the details of the dark area of ​​the image.

[0145] The above technical solution has the beneficial effect of grayscale processing of the pathological image to be identified, which can reduce the dimension of the image data and simplify the image processing steps, while retaining sufficient information for subsequent processing. A pixel point in the grayscale image is randomly selected as the first pixel point; the first image to be processed is determined with the first pixel point as the center and the first preset distance as the radius; the grayscale value of each pixel point in the first image to be processed is obtained and the grayscale mean is calculated to obtain the first grayscale mean; the absolute value of the difference between the first grayscale mean and the median is calculated to obtain the difference parameter; the grayscale value in the area of ​​the first image to be processed is determined. The method comprises the following steps: selecting a pixel point that is smaller than the screening parameter and generating a second image to be processed; obtaining the grayscale value of each pixel point in the third image to be processed and calculating the grayscale mean to obtain a second grayscale mean; determining the maximum grayscale value and the minimum grayscale value of the pixel points in the third image to be processed, calculating the difference between the maximum grayscale value and the minimum grayscale value, and obtaining the grayscale difference; randomly selecting a pixel point in the second image to be processed as the second pixel point; determining the third image to be processed with the second pixel point as the center and the second preset distance as the radius; the second preset distance is smaller than the first preset distance; based on the grayscale value of the second pixel point, the first preset distance and the second preset distance, the third image to be processed is obtained. The second grayscale mean and grayscale difference determine the enhancement multiple of the second pixel based on a preset algorithm; the second preset distance is less than the first preset distance, so as to screen out the pixel points that meet the third image to be processed in the second image to be processed, thereby improving the accuracy of the data; firstly, the first image to be processed is determined, and the screening parameters are obtained by comparing the difference parameters in the first image to be processed with the preset difference threshold, and the pixel points of the first image to be processed are screened by the screening parameters to complete the first screening of the pixels to be enhanced; the grayscale value, the second grayscale mean and the grayscale difference of the second pixel in the third image to be processed are calculated, and the enhancement multiple of the second pixel is determined based on the preset algorithm; in the pathological image data, since the brightness value of the dark area is relatively low, more image details cannot be observed and recorded, so the pixels in the dark area are enhanced; the present scheme is used to enhance the pathological image to be identified, and the area with relatively high brightness value is not enhanced, and only the pixels in the dark area are enhanced, which reduces the amount of calculation in the image enhancement process and improves the rate of image enhancement. Compared with the enhancement of the entire pathological image, more details of the original pathological image to be processed are retained, thereby improving the enhancement effect of the pathological image to be processed.

[0146] See also Figure 2 , which shows a structural block diagram of a cancer auxiliary detection system based on a triplet network of the present application.

[0147] like Figure 2 As shown, the pathological image recognition system 200 includes an acquisition module 210 , a training module 220 , a recognition module 230 and a discrimination module 240 .

[0148] The acquisition module 210 is configured to acquire a sample pathology image set, wherein the sample pathology image set includes an anchor point sample pathology image subset, a positive sample pathology image subset, and a negative sample pathology image subset;

[0149] The training module 220 is configured to iteratively train the triplet network model with the goal of minimizing the loss function of the triplet network model by using the anchor sample pathological image subset, the positive sample pathological image subset and the negative sample pathological image subset until the loss value of the loss function is less than or equal to a preset loss value threshold, or until the number of iterations of the triplet network model is greater than or equal to a number threshold, to obtain a pathological image recognition model;

[0150] The recognition module 230 is configured to input the acquired pathological image to be recognized into the pathological image recognition model, and the pathological image recognition model outputs a pathological area map containing at least one pathological feature;

[0151] The discrimination module 240 is configured to input the pathological region image containing at least one pathological feature into a preset pathological type discrimination model, and the pathological type discrimination model outputs a pathological type prediction result corresponding to the pathological image to be identified.

[0152] It should be understood that Figure 2 Modules and references documented in Figure 1 Therefore, the operations and features described above for the method and the corresponding technical effects are also applicable to Figure 2 The modules in it will not be described in detail here.

[0153] In other embodiments, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the program instructions are executed by a processor, the processor is caused to execute the cancer auxiliary detection method based on a triplet network in any of the above method embodiments;

[0154] As an implementation mode, the computer-readable storage medium of the present invention stores computer-executable instructions, and the computer-executable instructions are configured as follows:

[0155] Acquire a sample pathology image set, wherein the sample pathology image set includes an anchor sample pathology image subset, a positive sample pathology image subset, and a negative sample pathology image subset;

[0156] Taking minimization of the loss function of the triple network model as the goal, the triple network model is iteratively trained using the anchor sample pathological image subset, the positive sample pathological image subset and the negative sample pathological image subset until the loss value of the loss function is less than or equal to a preset loss value threshold, or until the number of iterations of the triple network model is greater than or equal to a number threshold, to obtain a pathological image recognition model;

[0157] Inputting the acquired pathological image to be identified into the pathological image recognition model, and the pathological image recognition model outputs a pathological area map containing at least one pathological feature;

[0158] A pathological region image including at least one pathological feature is input into a preset pathological type discrimination model, and the pathological type discrimination model outputs a pathological type prediction result corresponding to the pathological image to be identified.

[0159] The computer-readable storage medium may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to the use of a cancer auxiliary detection system based on a triplet network, etc. In addition, the computer-readable storage medium may include a high-speed random access memory, and may also include a memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the computer-readable storage medium may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the cancer auxiliary detection system based on the triplet network via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0160] Figure 3 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 3 As shown, the device includes: a processor 310 and a memory 320. The electronic device may also include: an input device 330 and an output device 340. The processor 310, the memory 320, the input device 330 and the output device 340 may be connected via a bus or other means. Figure 3 In the example, the connection through the bus is taken as an example. The memory 320 is the above-mentioned computer-readable storage medium. The processor 310 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 320, that is, the cancer auxiliary detection method based on the triple network of the above-mentioned method embodiment is realized. The input device 330 can receive input digital or character information, and generate key signal input related to user settings and function control of the cancer auxiliary detection system based on the triple network. The output device 340 may include a display device such as a display screen.

[0161] The electronic device can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0162] As an implementation mode, the electronic device is applied to a cancer auxiliary detection system based on a triplet network, and is used for a client, and includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can:

[0163] Acquire a sample pathology image set, wherein the sample pathology image set includes an anchor sample pathology image subset, a positive sample pathology image subset, and a negative sample pathology image subset;

[0164] Taking minimization of the loss function of the triple network model as the goal, the triple network model is iteratively trained using the anchor sample pathological image subset, the positive sample pathological image subset and the negative sample pathological image subset until the loss value of the loss function is less than or equal to a preset loss value threshold, or until the number of iterations of the triple network model is greater than or equal to a number threshold, to obtain a pathological image recognition model;

[0165] Inputting the acquired pathological image to be identified into the pathological image recognition model, and the pathological image recognition model outputs a pathological area map containing at least one pathological feature;

[0166] A pathological region image including at least one pathological feature is input into a preset pathological type discrimination model, and the pathological type discrimination model outputs a pathological type prediction result corresponding to the pathological image to be identified.

[0167] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cancer auxiliary detection system based on triplet network, characterized in that: include: An acquisition module is configured to acquire a sample pathology image set, wherein the sample pathology image set includes an anchor point sample pathology image subset, a positive sample pathology image subset, and a negative sample pathology image subset; A training module is configured to iteratively train the triplet network model with the goal of minimizing the loss function of the triplet network model by using the anchor sample pathological image subset, the positive sample pathological image subset and the negative sample pathological image subset until the loss value of the loss function is less than or equal to a preset loss value threshold, or until the number of iterations of the triplet network model is greater than or equal to a number threshold, to obtain a pathological image recognition model; A recognition module, configured to input the acquired pathological image to be recognized into the pathological image recognition model, and the pathological image recognition model outputs a pathological area map containing at least one pathological feature; The discrimination module is configured to input a pathological region image containing at least one pathological feature into a preset pathological type discrimination model, and the pathological type discrimination model outputs a pathological type prediction result corresponding to the pathological image to be identified.

2. The cancer auxiliary detection system based on triplet network according to claim 1, characterized in that: The system further comprises: The construction module is configured to preprocess each sample pathology image, and construct a sample pathology image set based on the type of each preprocessed sample pathology image.

3. The cancer auxiliary detection system based on triplet network according to claim 2, characterized in that: The construction module includes a cropping unit, which is used to crop the sample pathology image to obtain a cropped sample pathology image; The cutting unit comprises: Get subunits for: Obtaining the pixel value of each pixel in the sample pathological image; Determine a gradient value for each pixel based on the pixel value of each pixel; Compare the gradient value of each pixel with a preset gradient threshold, determine the pixel whose gradient value is greater than the preset gradient threshold as a first edge pixel, and obtain a first edge pixel set; determine a first edge image based on the first edge pixel set; Convert the first edge image into grayscale to obtain a grayscale image; Get the gray value of each pixel in the gray image; Generate a grayscale histogram based on the grayscale value of each pixel in the grayscale image; Determine the grayscale value range on both sides of the main peak of the area where the main peak is located according to the grayscale histogram, and determine the density value corresponding to each grayscale value in the grayscale value range on both sides of the main peak based on the grayscale value range on both sides of the main peak and the number of pixels corresponding to each grayscale value in the grayscale value range on both sides of the main peak; Calculate the Euclidean distance between pixels corresponding to two adjacent density values; Calculate the discreteness of each pixel based on the Euclidean distance between two adjacent density values ​​and the corresponding pixel points; determine the pixel points whose discreteness is less than a preset discreteness threshold as discrete points; Removing discrete points from the first edge image to obtain a second edge image; Performing image matching on the second edge image and the sample pathology image, and superimposing the second edge image and the sample pathology image according to the matching result; The cropping subunit is used to perform cropping according to the second image to obtain a cropped sample pathology image.

4. The cancer auxiliary detection system based on triplet network according to claim 1, characterized in that: in, The calculation process of the loss value of the loss function includes: Calculating a first distance corresponding to the triplet network model according to the anchor sample image features of the anchor sample pathological image and the positive sample image features of the positive sample pathological image, wherein the first distance represents a distance between a reference image feature and a positive sample image feature in the triplet network model; Calculating a second distance corresponding to the triplet network model according to the anchor sample image features of the anchor sample pathological image and the negative sample image features of the negative sample pathological image, wherein the second distance represents a distance between a reference image feature and a negative sample image feature in the triplet network model; A loss value of the loss function is determined according to the first distance and the second distance.

5. The cancer auxiliary detection system based on triplet network according to claim 1, characterized in that: The identification module comprises: A preprocessing unit, configured to preprocess the acquired pathological image to be identified to obtain a target pathological image to be identified, wherein the preprocessing includes cropping the sample pathological image, adjusting the scale of the sample pathological image, and color standardization of the sample pathological image; An extraction unit, configured to extract a feature region of the target pathological image to be identified based on a feature extractor, to obtain a feature region map corresponding to the target pathological image to be identified; The output unit is configured to input the feature area map corresponding to the target pathological image to be identified into the pathological image recognition model, and the pathological image recognition model outputs a pathological area map containing at least one pathological feature.

6. The cancer auxiliary detection system based on triplet network according to claim 5, characterized in that: in, The feature area map only includes at least one feature area.

7. The cancer auxiliary detection system based on triplet network according to claim 5, characterized in that: The feature extractor is a single-step multi-box object detection model.

8. The cancer auxiliary detection system based on triplet network according to claim 1, characterized in that: The system further comprises: an acquisition module, configured to acquire the at least one pathological feature and annotation data corresponding to the at least one pathological feature; The iteration module is configured to train the neural network model according to the at least one pathological feature and the labeled data corresponding to the at least one pathological feature until the number of iterations of the neural network model is greater than or equal to a number threshold, thereby obtaining a pathological type discrimination model.

9. The cancer auxiliary detection system based on triplet network according to claim 1, characterized in that: The system further comprises: an enhancement module, which is used to perform enhancement processing on the pathological image to be identified before the recognition module inputs the acquired pathological image to be identified into the pathological image recognition model; The enhancement module comprises: Computing unit for: Perform grayscale conversion on the pathological image to be identified to obtain a grayscale image; Randomly select a pixel point in the grayscale image as a first pixel point; determine a first image to be processed with the first pixel point as the center and a first preset distance as the radius; Obtaining the grayscale value of each pixel in the first image to be processed and calculating the grayscale mean to obtain a first grayscale mean; Obtaining the median value of each pixel in the first image to be processed; Calculate the absolute value of the difference between the first grayscale mean and the median to obtain a difference parameter; Compare the difference parameter with a preset difference threshold, and when it is determined that the difference parameter is greater than or equal to the preset difference threshold, obtain the smaller value between the first grayscale mean and the median value to obtain a screening parameter; Determine the pixel points whose grayscale values ​​are less than the screening parameter in the first image area to be processed, and generate a second image to be processed; Randomly select a pixel point in the second image to be processed as the second pixel point; determine the third image to be processed with the second pixel point as the center and the second preset distance as the radius; the second preset distance is smaller than the first preset distance; Obtaining the grayscale value of each pixel in the third image to be processed and calculating the grayscale mean to obtain a second grayscale mean; Determine the maximum grayscale value and the minimum grayscale value of the pixels in the third image to be processed, calculate the difference between the maximum grayscale value and the minimum grayscale value, and obtain the grayscale difference; Determine the enhancement multiple of the second pixel based on a preset algorithm based on the gray value of the second pixel, the second gray mean and the gray difference; Traversing each pixel in the third image to be processed, and determining an enhancement multiple for each pixel in the third image to be processed; Based on the enhancement multiple of each pixel in the third image to be processed, the third image to be processed is enhanced to obtain an enhanced pathological image to be identified; The output unit is used to input the enhanced pathological image to be identified into the pathological image recognition model.

10. A cancer auxiliary detection method based on triplet network, characterized in that: include: Acquire a sample pathology image set, wherein the sample pathology image set includes an anchor sample pathology image subset, a positive sample pathology image subset, and a negative sample pathology image subset; Taking minimization of the loss function of the triple network model as the goal, the triple network model is iteratively trained using the anchor sample pathological image subset, the positive sample pathological image subset and the negative sample pathological image subset until the loss value of the loss function is less than or equal to a preset loss value threshold, or until the number of iterations of the triple network model is greater than or equal to a number threshold, to obtain a pathological image recognition model; Inputting the acquired pathological image to be identified into the pathological image recognition model, and the pathological image recognition model outputs a pathological area map containing at least one pathological feature; A pathological region image including at least one pathological feature is input into a preset pathological type discrimination model, and the pathological type discrimination model outputs a pathological type prediction result corresponding to the pathological image to be identified.

Images