Automatic recognition method and system of pathological images based on deep learning

Abstract

The present invention discloses a method and system for automatic recognition of pathological images based on deep learning, and the present invention relates to the technical field of automatic recognition of tissue pathological images. The method comprises the following steps: collecting a number of digital pathological slice images as sample data, segmenting the collected sample image data, the segmented cell images include positive cell images and negative cell images, and storing the segmented positive cell images and negative cell images into positive cell image sets and negative cell image sets respectively; preprocessing the images in the two image sets to facilitate subsequent recognition and extraction of single cell images in the images; obtaining the extracted single cell images, extracting features of the single cell images to be identified, and training the initial neural network with the corresponding cell features as labels; generating a comprehensive scoring coefficient according to the cell features corresponding to the cell images in each region, and judging the detailed cell type according to the comprehensive scoring coefficient.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic recognition of tissue pathology images, and specifically to a method and system for automatic recognition of pathology images based on deep learning. Background Art

[0002] In the process of tumor diagnosis and prognosis, pathological diagnosis is one of the most commonly used methods, and the diagnosis results directly determine the formulation of treatment plans. In the process of pathological diagnosis, doctors need to observe pathological sections under a microscope, which is a lot of work and has a certain degree of subjectivity, and the diagnosis results are prone to subjective differences.

[0003] Digital pathology diagnosis is the application of information technology in the field of pathology. It is an organic combination of computer systems and traditional optical microscopes. Although digital pathology diagnosis technology improves diagnostic efficiency by changing the storage and reading format of pathology sections, the diagnostic results are still affected by subjective factors. More importantly, when reading digital sections, pathologists still need to continuously observe different areas of the same section in a fixed-size field of view, which takes a lot of time to complete the diagnosis.

[0004] With the development of artificial neural networks, deep learning has been increasingly used in the field of image processing. Its effect in image recognition, such as face recognition, has surpassed traditional pattern recognition and image processing algorithms. The application of deep learning algorithms in medical images has broad prospects. Deep learning uses layered neural networks to extract features, avoiding the subjective factors introduced by artificial feature extraction; it uses the back-propagation algorithm to update the network coefficients, so that the training results automatically converge to the optimal solution; and it enhances the generalization ability of the network by mining information from huge amounts of input data.

[0005] In the prior art, the publication number CN114118258A discloses a pathological slice feature fusion method based on a background-guided attention mechanism, which extracts the features of the pathological image slice using a pre-trained deep neural network and converts it into a d-dimensional feature using a linear transformation; randomly generates a d-dimensional initial background vector according to the d-dimensional normal distribution; starting from the generated initial background vector, after a loop module, the background vector of the pathological image is obtained; using the obtained background vector, the attention weight of each slice feature is calculated; the features are weighted and summed according to their attention weights to obtain the fused pathological image features. This method improves the accuracy of the fused features, effectively improves the quality of the fused pathological image features, and significantly improves the accuracy of subsequent classification tasks, but the initial background vector is randomly generated according to the d-dimensional normal distribution. This method may cause the initial value of the background vector to have a large uncertainty, thereby affecting the stability and consistency of the entire model, resulting in a decrease in the accuracy and effectiveness of the model in performing recognition tasks.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0007] The purpose of the present invention is to provide a method and system for automatic recognition of pathological images based on deep learning to solve the problems raised in the above-mentioned background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] The automatic recognition method of pathological images based on deep learning includes the following specific steps:

[0010] Acquiring a plurality of digital pathology slice images, wherein the digital pathology slice images are immunohistochemically stained digital slice images;

[0011] Performing image segmentation on the collected immunohistochemically stained digital slice images, segmenting the immunohistochemically stained digital slice images into positive cell images, negative cell images and background images, generating a positive cell image set and a negative cell image set, and storing the segmented positive cell images and negative cell images into the positive cell image set and the negative cell image set respectively;

[0012] Preprocessing the cell images in the positive cell image set and the negative cell image set, wherein the preprocessing includes image noise reduction processing and enhancement processing;

[0013] A cell extraction model is constructed based on the YOLOv5s network, and the preprocessed cell images are input into the cell extraction model to generate a plurality of single cell images of different abstract cell types, wherein the abstract cell types include independent cells and double cell adhesion cell blocks;

[0014] Using the extracted single cell image as a sample, extracting features from the single cell image, wherein the features include the perimeter, area, shape factor and color of the cell, marking the extracted features as labels on the single cell image, and using the single cell image marked with the labels to train an initial neural network to obtain a cell feature extraction model;

[0015] After the immunohistochemically stained digital slice image to be identified is segmented and preprocessed according to the above method, it is input into the cell extraction model to obtain a single cell image to be identified, and the single cell image to be identified is input into the trained cell feature extraction model to obtain the cell features corresponding to the single cell image to be identified;

[0016] A comprehensive scoring coefficient is generated according to the cell features corresponding to the single cell image to be identified, and the generated comprehensive scoring coefficient is compared with a preset cell type threshold. The detailed cell type is determined according to different comparison results. The detailed cell types include independent positive cells, double cell adhesion positive cell blocks, independent negative cells and double cell adhesion negative cell blocks.

[0017] Furthermore, several digital pathology slice images are collected, wherein the digital pathology slice images are immunohistochemically stained digital slice images, wherein the immunohistochemically stained digital slice images use proliferating cell nuclear antigen PCNA as a marker, and stain tumor cells in different cell cycles by immunohistochemistry, and the substrate used is 3,3'-diaminobenzidine, so that all PCNA-positive cell nuclei appear brown-stained, while negative cell nuclei will not be stained.

[0018] Furthermore, the collected immunohistochemically stained digital slice images are segmented, wherein the image segmentation method is: a two-dimensional maximum entropy threshold segmentation method based on two color models, and the formula for the two-dimensional entropy of the image is:

[0019]

[0020] In the formula, is the two-dimensional entropy of the image, Represents the probability of a point with grayscale value 𝑖 and neighborhood grayscale mean 𝑗 appearing in the image; Set the threshold vector , represents the local gray value, Represents the pixel area mean, for grayscale image, calibrated at the threshold Define the probability of regions 1 and 2 as and , the regions 1 and 2 are the target cell region and the background region, respectively. and The formula is:

[0021]

[0022]

[0023] The formula for the discriminant function of two-dimensional entropy is:

[0024]

[0025] In the formula, is the discriminant function of two-dimensional entropy, and denote the two-dimensional entropy of region 1 and region 2 respectively, where and The formula is:

[0026]

[0027]

[0028] In the formula, is the image grayscale.

[0029] Furthermore, the target cell region obtained by the two-dimensional maximum entropy threshold segmentation method is distinguished by the RGB model into positive cell images and negative cell images, and the segmented positive cell images and negative cell images are correspondingly stored in the positive cell image set and the negative cell image set.

[0030] Furthermore, the cell images in the positive cell image set and the negative cell image set are preprocessed, and the logic of the preprocessing is as follows:

[0031] The denoising method of wavelet transform is used to denoise the cell image. The specific steps of the wavelet transform denoising method include: decomposing the cell image through wavelet transform to obtain the wavelet coefficients of the image at different scales and directions; thresholding the wavelet coefficients to set the low-amplitude wavelet coefficients to zero and retain the high-amplitude wavelet coefficients; inversely transforming the wavelet coefficients after thresholding, reconstructing the processed coefficients into an image, and completing the denoising of the cell image;

[0032] Bilateral filtering is used to enhance the details of the cell image. The formula for the specific filtering transformation is:

[0033]

[0034] In the formula, is the coordinate vector in the image coordinate space, is the coordinate vector The gray value at Gray value The gray value after bilateral filtering transformation is are all Gaussian functions, where The formula is:

[0035]

[0036]

[0037] In the formula, is the coordinate vector in the image coordinate space, is the coordinate vector The gray value at and They are The standard deviation of .

[0038] Furthermore, a cell extraction model is constructed based on the YOLOv5s network, and the preprocessed cell image is input into the cell extraction model to generate a plurality of single cell images of different abstract cell types, wherein the logic for generating a plurality of single cell images of different abstract cell types is as follows:

[0039] A plurality of public cell images are collected as a sample set, and the public cell images contain independent cells and double-cell adhesion cell blocks. The edge areas of the independent cells and double-cell adhesion cell blocks in the public cell images are annotated by manual labeling, and then input into a YOLOv5s network for training to obtain a cell extraction model. The input of the cell extraction model is the cell image, and the output is single cell images of different abstract cell types, wherein the abstract cell types include independent cells and double-cell adhesion cell blocks. The preprocessed cell images are then input into the cell extraction model to generate a plurality of single cell images of different abstract cell types.

[0040] Furthermore, feature extraction is performed on the single cell image to be identified, wherein the features include the perimeter, area, shape factor and color of the cell, wherein the shape factor is calculated based on the formula:

[0041]

[0042] In the formula, is the shape factor of the cell, represents the area of ​​the cell, is the perimeter of the cell;

[0043] The logic for assigning cell colors is: a positive cell nucleus with brown staining is assigned a value of 1, and a negative cell nucleus without staining is assigned a value of 0. The initial neural network is trained to obtain a cell feature extraction model, wherein the cell feature extraction model is established based on a convolutional neural network.

[0044] Furthermore, a comprehensive scoring coefficient is generated according to the cell features corresponding to the single cell image to be identified, wherein the logic for generating the comprehensive scoring coefficient is:

[0045]

[0046] In the formula, is the comprehensive scoring coefficient, and are the weight coefficients of the cell area, cell perimeter and cell shape factor, respectively, where and are greater than 0 and ;

[0047] The generated comprehensive scoring coefficient is compared with the pre-set cell type threshold, and the logic for determining the detailed cell type based on different comparison results is as follows: the color of the calibrated cell is assigned as , the pre-set cell type threshold is ;

[0048] when and When , the cell is judged as an independent positive cell;

[0049] when and When , the cell is judged as an independent negative cell;

[0050] when and When the cell is positive, it is judged as a double cell adhesion positive cell mass;

[0051] when and When the cell is detected by immunohistochemistry, it is judged that the cell is a negative cell mass of double cell adhesion.

[0052] The present invention also provides a pathological image automatic recognition system based on deep learning, and the pathological image automatic recognition system based on deep learning is used to execute the above-mentioned pathological image automatic recognition method based on deep learning, including:

[0053] A sample data acquisition module is used to acquire a plurality of digital pathology slice images, wherein the digital pathology slice images are immunohistochemically stained digital slice images;

[0054] An image segmentation module is used to perform image segmentation on the collected immunohistochemically stained digital slice images, segment the original immunohistochemically stained digital slice images into positive cell images, negative cell images and background images, generate positive cell image sets and negative cell image sets, and store the segmented positive cell images and negative cell images into the positive cell image set and the negative cell image set respectively;

[0055] An image preprocessing module, used for preprocessing the cell images in the positive cell image set and the negative cell image set, wherein the preprocessing includes image noise reduction processing and enhancement processing;

[0056] A target extraction module is used to build a cell extraction model based on the YOLOv5s network, and input the preprocessed cell image into the cell extraction model to generate a plurality of single cell images of different abstract cell types, wherein the abstract cell types include independent cells and double cell adhesion cell blocks;

[0057] A feature extraction model training module is used to use the extracted single cell image as a sample to extract features from the single cell image, wherein the features include the perimeter, area, shape factor and color of the cell, and to mark the extracted features as labels on the single cell image, and to train the initial neural network using the single cell image marked with the labels to obtain a cell feature extraction model;

[0058] A feature extraction module is used to segment and preprocess the immunohistochemically stained digital slice image to be identified according to the above method, and then input it into the cell extraction model to obtain a single cell image to be identified, and input the single cell image to be identified into the trained cell feature extraction model to obtain the cell features corresponding to the single cell image to be identified;

[0059] The pathological cell identification module is used to generate a comprehensive scoring coefficient according to the cell features corresponding to each single cell image to be identified, compare the generated comprehensive scoring coefficient with a pre-set cell type threshold, and determine the detailed cell type according to different comparison results. The detailed cell type includes independent positive cells, positive cell blocks with double cell adhesion, independent negative cells, and negative cell blocks with double cell adhesion.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] Several digital pathology slice images were collected as sample data, and the two-dimensional maximum entropy threshold segmentation method was used for the collected sample image data to separate the cell image and the background image in the cell sample image. The segmented cell images were then distinguished from the positive cell image and the negative cell image by the RGB model to generate a positive cell image set and a negative cell image set. The segmented positive cell images and negative cell images were correspondingly stored in the positive cell image set and the negative cell image set; the images in the positive cell image set and the negative cell image set were preprocessed to improve the image quality and enhance the edges of individual cells; a cell extraction model was constructed based on the YOLOv5s network, and the preprocessed The processed cell image is input into the cell extraction model to generate a plurality of single cell images of different abstract cell types, wherein the abstract cell types include independent cells and double cell adhesion cell blocks, and the extracted single cell images are used as samples to extract features of the cell images of the region to be identified, and the corresponding cell features are used as labels to train the initial neural network; the image to be identified is segmented, preprocessed and the target is extracted as the input of the trained neural network, the cell features corresponding to the single cell image to be identified are obtained, and a comprehensive scoring coefficient is generated according to the cell features corresponding to the single cell image to be identified, and the detailed cell type is judged according to the comprehensive scoring coefficient. This method firstly segments and preprocesses the image data to enhance the image quality, so that the accuracy of the recognition model is improved, and then the convolutional neural network is used to extract different types of cell features, and the comprehensive scoring coefficient is generated from the corresponding cell features, and the cell type is judged according to the comprehensive scoring coefficient, thereby improving the recognition stability and effectiveness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0063] Figure 2 It is a schematic diagram of the overall system structure of the present invention. DETAILED DESCRIPTION

[0064] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0065] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0066] Example:

[0067] See also Figure 1 , the present invention provides a technical solution:

[0068] The automatic recognition method of pathological images based on deep learning includes the following specific steps:

[0069] Step 1: Acquire a number of digital pathology slice images, wherein the digital pathology slice images are immunohistochemically stained digital slice images;

[0070] Several digital pathology slice images are collected, wherein the digital pathology slice images are immunohistochemically stained digital slice images, wherein the immunohistochemically stained digital slice images use proliferating cell nuclear antigen PCNA as a marker, and stain tumor cells in different cell cycles by immunohistochemistry, and the substrate used is 3,3'-diaminobenzidine, so that all PCNA-positive cell nuclei appear brown-stained, while negative cell nuclei will not be stained.

[0071] Proliferating cell nuclear antigen (PCNA) can be used to distinguish between tumor cells with strong proliferation ability and G0 phase cells, because proliferating cells express PCNA in the G1, S, G2, and M phases of the cell cycle, while PCNA expression in G0 phase cells is very low or not expressed. Cells in the G1, S, G2, and M phases of the cycle are positive cells, while cells in the G0 phase are negative cells.

[0072] Step 2: performing image segmentation on the collected immunohistochemically stained digital slice images, segmenting the original immunohistochemically stained digital slice images into positive cell images, negative cell images and background images, generating a positive cell image set and a negative cell image set, and storing the segmented positive cell images and negative cell images into the positive cell image set and the negative cell image set respectively;

[0073] The collected digital slice images of immunohistochemical staining were segmented, wherein the image segmentation method was: a two-dimensional maximum entropy threshold segmentation method based on two color models, and the formula for the two-dimensional entropy of the image was:

[0074]

[0075] In the formula, is the two-dimensional entropy of the image, Represents the probability of a point with grayscale value 𝑖 and neighborhood grayscale mean 𝑗 appearing in the image; Set the threshold vector , represents the local gray value, Represents the pixel area mean, for grayscale image, calibrated at the threshold Define the probability of regions 1 and 2 as and , the regions 1 and 2 are the target cell region and the background region, respectively. and The formula is:

[0076]

[0077]

[0078] The formula for the discriminant function of two-dimensional entropy is:

[0079]

[0080] In the formula, is the discriminant function of two-dimensional entropy, and denote the two-dimensional entropy of region 1 and region 2 respectively, where and The formula is:

[0081]

[0082]

[0083] In the formula, is the image grayscale.

[0084] The two-dimensional maximum entropy threshold method uses the gray value of the pixel point and the neighborhood gray mean to generate a two-dimensional gray histogram, where the horizontal axis is the local gray, the vertical axis is the neighborhood gray mean, and the area far from the diagonal is noise. The optimal two-dimensional threshold vector is obtained by traversing the gray distribution space. To distinguish the target from the background, the optimal two-dimensional threshold vector is selected By maximizing the entropy discriminant function, the difference between positive and negative cells and between cells and background can be guaranteed during segmentation. The two-dimensional maximum entropy threshold effectively solves the influence of signal-to-noise ratio and target-to-background ratio on the segmentation algorithm, and is suitable for complex and diverse pathological images.

[0085] The target cell area obtained by the two-dimensional maximum entropy threshold segmentation method is distinguished by the RGB model into positive cell images and negative cell images, and the segmented positive cell images and negative cell images are stored in the positive cell image set and the negative cell image set respectively; the RGB model is the most commonly used color model. In the optical scanning system, the color is represented by a linear combination of three components. The RGB model is used to analyze pathological images from the perspective of natural color composition, which can accurately distinguish positive cells from negative cells.

[0086] Step 3: preprocessing the cell images in the positive cell image set and the negative cell image set, wherein the preprocessing includes image noise reduction processing and enhancement processing;

[0087] The wavelet transform denoising method is used to denoise the medical test report image.

[0088] Among them, the denoising method of wavelet transform is used to denoise the cell image. The specific steps of the wavelet transform denoising method include: converting the table area of ​​the medical test form into a grayscale image, normalizing the image so that the grayscale value is between 0 and 1; selecting a suitable wavelet basis function and determining the wavelet function (such as Daubechies wavelet db4); selecting the number of layers of wavelet decomposition, generally 3 to 5 layers; thresholding the wavelet coefficients, selecting the threshold method, the threshold method includes hard threshold or soft threshold, and calculating the threshold; performing inverse wavelet transform on the wavelet coefficients after thresholding, and reconstructing the denoised image; wherein the threshold is calculated using the VisuShrink method, and the specific calculation formula is:

[0089]

[0090] In the formula, is the threshold value, is the standard deviation of the noise, is the total number of pixels in the image;

[0091] Bilateral filtering is used to enhance the details of the cell image. The formula for the specific filtering transformation is:

[0092]

[0093] In the formula, is the coordinate vector in the image coordinate space, is the coordinate vector The gray value at Gray value The gray value after bilateral filtering transformation is are all Gaussian functions, where The formula is:

[0094]

[0095]

[0096] In the formula, is the coordinate vector in the image coordinate space, is the coordinate vector The gray value at and They are The standard deviation of .

[0097] It measures the spatial relationship between x and y, so that pixels closer to each other have a greater impact on the grayscale value at x; It measures the relationship between the value ranges of x and y, so that points with similar grayscale values ​​in the neighborhood have greater weights in the calculation; and are all Gaussian functions, with standard deviations of and In flat areas, the grayscale value changes little in all directions, and the range weight In the neighborhood, it is close to 1. At this time, only the position relationship affects the filtering result. It is equivalent to Gaussian filtering and has a good suppressive effect on noise. At the edge of the cell, the grayscale value changes dramatically in the normal direction of the boundary, causing the value range weight to drop sharply. Only the pixels in the adjacent neighborhood along the tangent direction of the boundary affect the center point, thereby strengthening the edge information of the cell.

[0098] Step 4: construct a cell extraction model based on the YOLOv5s network, and input the preprocessed cell image into the cell extraction model to generate a plurality of single cell images of different abstract cell types, wherein the abstract cell types include independent cells and double cell adhesion cell blocks;

[0099] A cell extraction model is constructed based on the YOLOv5s network, and the preprocessed cell images are input into the cell extraction model to generate multiple single cell images of different abstract cell types. The logic for generating multiple single cell images of different abstract cell types is as follows:

[0100] A plurality of public cell images are collected as a sample set, and the public cell images contain independent cells and double-cell adhesion cell blocks. The edge areas of the independent cells and double-cell adhesion cell blocks in the public cell images are annotated by manual labeling, and then input into a YOLOv5s network for training to obtain a cell extraction model. The input of the cell extraction model is the cell image, and the output is single cell images of different abstract cell types, wherein the abstract cell types include independent cells and double-cell adhesion cell blocks. The preprocessed cell images are then input into the cell extraction model to generate a plurality of single cell images of different abstract cell types.

[0101] Among them, the main function of the YOLOv5s network is to perform real-time target detection. Target detection is an important task in the field of computer vision. Its goal is to detect various types of objects from images or videos and accurately mark their locations. YOLOv5s is a model in the YOLO (You Only Look Once) series, which strikes a balance between real-time and accuracy;

[0102] After the cell targets are extracted, all cell targets in the image are labeled, and a VOC format dataset is created using the LabelImg tool. The VOC format dataset is then converted into a YOLO format dataset suitable for the YOLOv5s network, including independent cells and double-cell adhesion cell blocks.

[0103] The labeled cell target image is cropped, and the cropping method is: cropping based on the minimum bounding rectangle, and the specific steps include: using the detection algorithm YOLOv5s network and the LabelImg tool to label the bounding box of a single cell; obtaining the coordinates of the minimum bounding rectangle of the labeled cell target, including the coordinates of the upper left corner boundary point and the coordinates of the lower right corner boundary point of the minimum bounding rectangle; and cropping different types of cell images containing complete single cells according to the coordinates of the minimum bounding rectangle of the cell target.

[0104] Step 5: taking the extracted single cell image as a sample, extracting features from the single cell image, wherein the features include the perimeter, area, shape factor and color of the cell, marking the extracted features as labels on the single cell image, and using the single cell image marked with the labels to train the initial neural network to obtain a cell feature extraction model;

[0105] The cell feature extraction model is composed of a convolutional neural network, which includes an input layer, a convolution layer, an activation function layer, a pooling layer, a fully connected layer, a normalization layer and an output layer, wherein the activation function layer selects the ReLU function as the activation function;

[0106] Feature extraction is performed on a single cell image, wherein the features include the cell's perimeter, area, shape factor, and color, wherein the shape factor is calculated based on the formula:

[0107]

[0108] In the formula, is the shape factor of the cell, represents the area of ​​the cell, is the perimeter of the cell;

[0109] The logic for assigning cell colors is: a positive cell nucleus with brown staining is assigned a value of 1, and a negative cell nucleus without staining is assigned a value of 0. The initial neural network is trained to obtain a cell feature extraction model, wherein the cell feature extraction model is established based on a convolutional neural network.

[0110] Step 6: After segmenting and preprocessing the immunohistochemically stained digital slice image to be identified according to the above method, the image is input into the cell extraction model to obtain a single cell image to be identified, and the single cell image to be identified is input into the trained cell feature extraction model to obtain the cell features corresponding to the single cell image to be identified;

[0111] For the sake of distinction, the processed image to be identified in real time is called a single cell image to be identified, and the above-mentioned image used for training is called a single cell picture.

[0112] Step 7: Generate a comprehensive scoring coefficient based on the cell features corresponding to the single cell image to be identified, compare the generated comprehensive scoring coefficient with the pre-set cell type threshold, and determine the detailed cell type based on different comparison results. The detailed cell types include independent positive cells, double cell adhesion positive cell blocks, independent negative cells, and double cell adhesion negative cell blocks.

[0113] A comprehensive scoring coefficient is generated according to the cell features corresponding to the single cell image to be identified, wherein the logic for generating the comprehensive scoring coefficient is:

[0114]

[0115] In the formula, is the comprehensive scoring coefficient, and are the weight coefficients of the cell area, cell perimeter and cell shape factor, respectively, where and are greater than 0 and ;in, The area of ​​the cell and the cell perimeter The larger the cell area, the greater the comprehensive score coefficient. The higher the cell's circumference The greater the score The higher the shape factor of the cell The smaller the value, the more complex and irregular the shape. Shape factor of cells Inversely proportional;

[0116] The generated comprehensive scoring coefficient is compared with the pre-set cell type threshold, and the logic for determining the detailed cell type based on different comparison results is as follows: the color of the calibrated cell is assigned as , the pre-set cell type threshold is ;

[0117] when and When , the cell is judged as an independent positive cell;

[0118] when and When , the cell is judged as an independent negative cell;

[0119] when and When the cell is positive, it is judged as a double cell adhesion positive cell mass;

[0120] when and When the cell is detected by immunohistochemistry, it is judged that the cell is a negative cell mass of double cell adhesion.

[0121] See also Figure 2 The present invention also provides a pathological image automatic recognition system based on deep learning, and the pathological image automatic recognition system based on deep learning is used to execute the above-mentioned pathological image automatic recognition method based on deep learning, including:

[0122] A sample data acquisition module is used to acquire a plurality of digital pathology slice images, wherein the digital pathology slice images are immunohistochemically stained digital slice images;

[0123] An image segmentation module is used to perform image segmentation on the collected immunohistochemically stained digital slice images, segment the original immunohistochemically stained digital slice images into positive cell images, negative cell images and background images, generate positive cell image sets and negative cell image sets, and store the segmented positive cell images and negative cell images into the positive cell image set and the negative cell image set respectively;

[0124] An image preprocessing module, used for preprocessing the cell images in the positive cell image set and the negative cell image set, wherein the preprocessing includes image noise reduction processing and enhancement processing;

[0125] A target extraction module is used to build a cell extraction model based on the YOLOv5s network, and input the preprocessed cell image into the cell extraction model to generate a plurality of single cell images of different abstract cell types, wherein the abstract cell types include independent cells and double cell adhesion cell blocks;

[0126] A feature extraction model training module is used to extract features from the single cell image after extraction as a sample, wherein the features include the perimeter, area, shape factor and color of the cell, and to train the initial neural network using the corresponding cell features as labels to obtain a cell feature extraction model;

[0127] A feature extraction module is used to segment, preprocess, extract targets and crop the immunohistochemically stained digital slice image to be identified, obtain a single cell image to be identified, input the single cell image to be identified into the trained cell feature extraction model, and obtain the cell features corresponding to the single cell image to be identified;

[0128] The pathological cell identification module is used to generate a comprehensive scoring coefficient according to the cell features corresponding to each single cell image to be identified, compare the generated comprehensive scoring coefficient with a pre-set cell type threshold, and determine the detailed cell type according to different comparison results. The detailed cell type includes independent positive cells, positive cell blocks with double cell adhesion, independent negative cells, and negative cell blocks with double cell adhesion.

[0129] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0130] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for automatic recognition of pathological images based on deep learning, characterized in that: The specific steps include: Collecting a number of digital pathology slice images, wherein the digital pathology slice images are immunohistochemically stained digital slice images, wherein the immunohistochemically stained digital slice images use proliferating cell nuclear antigen (PCNA) as a marker, and use immunohistochemistry to stain tumor cells in different cell cycles, and the substrate used is 3,3'-diaminobenzidine, so that all PCNA-positive cell nuclei are stained brown, while negative cell nuclei are not stained; Performing image segmentation on the collected immunohistochemically stained digital slice images, segmenting the immunohistochemically stained digital slice images into positive cell images, negative cell images and background images, generating a positive cell image set and a negative cell image set, and storing the segmented positive cell images and negative cell images into the positive cell image set and the negative cell image set respectively; The collected digital slice images of immunohistochemical staining were segmented, wherein the image segmentation method was: a two-dimensional maximum entropy threshold segmentation method based on two color models, and the formula for the two-dimensional entropy of the image was: Where H is the two-dimensional entropy of the image, P i,j Represents the probability of a point with gray value i and neighborhood gray mean j appearing in the image; set the threshold vector (k, q), k represents the local gray value, q represents the pixel domain mean, for an image with L gray levels, the probability of defining regions 1 and 2 at the threshold (k, q) is p1 and p2, where regions 1 and 2 are the target cell region and the background region, respectively, and the formulas for p1 and p2 are: The formula for the discriminant function of two-dimensional entropy is: In the formula, is the discriminant function of the two-dimensional entropy, H1 and H2 represent the two-dimensional entropy of region 1 and region 2 respectively, where the formulas based on H1 and H2 are: Where L is the gray level of the image; The target cell region obtained by the two-dimensional maximum entropy threshold segmentation method is distinguished into positive cell images and negative cell images through the RGB model, and the segmented positive cell images and negative cell images are correspondingly stored in the positive cell image set and the negative cell image set; Preprocessing the cell images in the positive cell image set and the negative cell image set, wherein the preprocessing includes image noise reduction processing and enhancement processing; The cell images in the positive cell image set and the negative cell image set are preprocessed, and the logic of the preprocessing is as follows: The denoising method of wavelet transform is used to denoise the cell image. The specific steps of the wavelet transform denoising method include: decomposing the cell image through wavelet transform to obtain the wavelet coefficients of the image at different scales and directions; thresholding the wavelet coefficients to set the low-amplitude wavelet coefficients to zero and retain the high-amplitude wavelet coefficients; inversely transforming the wavelet coefficients after thresholding, reconstructing the processed coefficients into an image, and completing the denoising of the cell image; Bilateral filtering is used to enhance the details of the cell image. The formula for the specific filtering transformation is: Where y is the coordinate vector in the image coordinate space, I y is the gray value at the coordinate vector y, B y is the gray value I y The gray value after bilateral filtering transformation, G d and G r are all Gaussian functions, where G d and G r The formula is: Where x is the coordinate vector in the image coordinate space, I x is the gray value at the coordinate vector x, σ d and σ r G d and G r The standard deviation of A cell extraction model is constructed based on the YOLOv5s network, and the preprocessed cell images are input into the cell extraction model to generate a plurality of single cell images of different abstract cell types, wherein the abstract cell types include independent cells and double cell adhesion cell blocks; A cell extraction model is constructed based on the YOLOv5s network, and the preprocessed cell images are input into the cell extraction model to generate multiple single cell images of different abstract cell types. The logic for generating multiple single cell images of different abstract cell types is as follows: Collect multiple publicly available cell images as a sample set. The publicly available cell images contain independent cells and double-cell adhesion cell clusters. After manually labeling the edge regions of the independent cells and double-cell adhesion cell clusters in the publicly available cell images, input them into the YOLOv5s network for training to obtain a cell extraction model. The input of the cell extraction model is a cell image, and the output is single-cell images of different general cell types. The general cell types include independent cells and double-cell adhesion cell clusters. Then, input the preprocessed cell image into the cell extraction model to generate single-cell images of multiple different general cell types; Use the extracted single-cell images as samples, extract features from the single-cell images. The features include the perimeter, area, shape factor, and color of the cells. Use the extracted features as labels to label the single-cell images, and use the single-cell images labeled with labels to train the initial neural network to obtain a cell feature extraction model; After segmenting and preprocessing the immunohistochemical staining digital slice image to be recognized according to the above method, input it into the cell extraction model to obtain the single-cell image to be recognized. Input the single-cell image to be recognized into the trained cell feature extraction model to obtain the cell features corresponding to the single-cell image to be recognized; Extract features from the single-cell image. The features include the perimeter, area, shape factor, and color of the cells. The formula for calculating the shape factor is: In the formula, SF is the shape factor of the cell, Area represents the area of the cell, and perimeter is the perimeter of the cell; The logic for assigning values to the color of the cells is as follows: Assign a value of 1 to the positive cell nucleus that appears brown-stained, and assign a value of 0 to the negative cell nucleus that is not stained. Train the initial neural network to obtain a cell feature extraction model, where the cell feature extraction model is established based on a convolutional neural network; Generate a comprehensive scoring coefficient based on the cell features corresponding to the single-cell image to be recognized. Compare the generated comprehensive scoring coefficient with the pre-set cell type threshold, and judge the detailed cell type according to different comparison results. The detailed cell types include independent positive cells, double-cell adhesion positive cell clusters, independent negative cells, and double-cell adhesion negative cell clusters; Generate a comprehensive scoring coefficient based on the cell features corresponding to the single-cell image to be recognized. The logic for generating the comprehensive scoring coefficient is: In the formula, score is the comprehensive scoring coefficient, ω1, ω2, and ω3 are the weight coefficients of the area of the cell, the perimeter of the cell, and the cell shape factor respectively. ω1, ω2, and ω3 are all greater than 0 and ω1 ≤ ω2 < ω3; Compare the generated comprehensive scoring coefficient with the pre-set cell type threshold, and the logic for judging the detailed cell type according to different comparison results is: The color of the calibrated cell is assigned as YS, and the pre-set cell type threshold is YZ; When 0 < score ≤ YZ * 0.4 and YS > 0, judge that the cell is an independent positive cell; When 0 < score ≤ YZ * 0.4 and YS = 0, determine that the cell is an independent negative cell; When YZ * 0.4 < score ≤ YZ * 1.0 and YS > 0, determine that the cell is a positive cell mass with double - cell adhesion; When YZ * 0.4 < score ≤ YZ * 1.0 and YS = 0, determine that the cell is a negative cell mass with double - cell adhesion.

2. A pathological image automatic recognition system based on deep learning, characterized by: The deep - learning - based pathological image automatic recognition system is used to execute the deep - learning - based pathological image automatic recognition method according to claim 1, including: A sample data acquisition module, which is used to acquire a number of digital pathological section images, and the digital pathological section images are immunohistochemical staining digital section images; An image segmentation module, which is used to perform image segmentation on the acquired immunohistochemical staining digital section images, segment the original immunohistochemical staining digital section image into positive cell images, negative cell images and background images, generate a positive cell image set and a negative cell image set, and store the segmented positive cell images and negative cell images into the positive cell image set and the negative cell image set correspondingly; An image pre - processing module, which is used to perform pre - processing on the cell images in the positive cell image set and the negative cell image set, and the pre - processing includes image noise reduction processing and enhancement processing; A target extraction module, which is used to construct a cell extraction model based on the YOLOv5s network, input the pre - processed cell images into the cell extraction model to generate single - cell pictures of multiple different general cell types, and the general cell types include independent cells and double - cell adhesion cell masses; A feature extraction model training module, which is used to take the extracted single - cell pictures as samples, extract features of the single - cell pictures, the features include the perimeter, area, shape factor and color of the cells, label the extracted features on the single - cell pictures, and use the single - cell pictures labeled with labels to train the initial neural network to obtain a cell feature extraction model; A feature extraction module, which is used to segment and pre - process the immunohistochemical staining digital section image to be recognized according to the above method, input it into the cell extraction model to obtain the single - cell image to be recognized, and input the single - cell image to be recognized into the trained cell feature extraction model to obtain the cell features corresponding to the single - cell image to be recognized; A pathological cell recognition module, which is used to generate a comprehensive scoring coefficient according to the cell features corresponding to each single - cell image to be recognized, compare the generated comprehensive scoring coefficient with the preset cell type threshold, and judge the detailed cell type according to different comparison results. The detailed cell types include independent positive cells, positive cell masses with double - cell adhesion, independent negative cells and negative cell masses with double - cell adhesion.

Images