Deep learning-based adenocarcinoma prediction system and model construction method

Abstract

The invention provides a deep learning-based adenocarcinoma prediction system and a model construction method. The method comprises the following steps: acquiring a module pathological section as a training sample; the amplification module amplifies the training sample; a segmentation module segments the amplified training sample to form a first training image block, and a labeling module labels each prediction item for the training image block in proportion to form a label vector; the training module uses a training block with a label vector to train an initial adenocarcinoma prediction model; and the prediction module uses the trained adenocarcinoma prediction model to predict prediction blocks segmented by a to-be-predicted pathological section, outputs predicted proportions of prediction items, and obtains a prediction result of the to-be-predicted pathological section according to the proportions of the prediction items of all the blocks. And the accuracy of adenocarcinoma prediction is improved by marking training according to the proportion.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to an adenocarcinoma prediction system and model building method based on deep learning. Background Art

[0002] Prostate cancer and breast cancer are malignant tumors with high morbidity and mortality. In clinical routine, pathologists diagnose cancer by observing the morphology of glands and cells in the patient's pathological sections with the naked eye, and score prostate sections and grade breast cancer sections according to the standards of the World Health Organization. Different scores / grades correspond to different treatments and prognosis plans. However, the number of pathologists cannot meet the large number of biopsy samples, so in recent years, deep learning models have been used for cancer scoring / grading to help doctors reduce their burden.

[0003] Due to the large resolution and memory usage of digital pathology sections, conventional graphics cards used for deep learning training are not sufficient to accommodate complete pathology sections. The common practice in the industry is to cut pathology sections into patches and input them into the model.

[0004] In order to train the model to recognize tiles, the common practice in the industry is to ask pathologists to label prostate tiles with a single grade (grading) as a label and breast cancer tiles with a single score (score) as a label. However, prostate cancer and breast cancer are different from other types of cancer. Prostate cancer and breast cancer are invasive. There are normal glands and various grades of cancer mixed in prostate cancer tiles. Therefore, it is inaccurate to use a single Gleason grade as a prostate tile label. There are tiles with arbitrary glandular cavity ratios, arbitrary atypical nuclear ratios, and arbitrary numbers of nuclear divisions in breast cancer slices. Therefore, it is also inaccurate to use a single score as a breast cancer tile label.

[0005] Moreover, the model trained based on a single grade / score as a tile annotation can only output a single grade / score for a tile in clinical application, and cannot correctly predict the grade / score when encountering a tile with multiple cancers mixed. If the grade / score in the tile cannot be accurately predicted, the deviation will be further amplified when the aggregated tile information is used to score / grade the pathological section, resulting in the inability to accurately score or grade the pathological section. Summary of the invention

[0006] In response to the above-mentioned problems, the present invention provides a deep learning-based adenocarcinoma prediction system and model construction method for adenocarcinoma, which solves the technical problem that the accuracy of adenocarcinoma prediction by models in the prior art needs to be improved.

[0007] A deep learning-based adenocarcinoma prediction system for prostate cancer scoring, including:

[0008] A first acquisition module, used for acquiring a pathological section of the prostate as a first training sample;

[0009] A first amplification module, connected to the first acquisition module, for amplifying the first training sample;

[0010] A first segmentation module, connected to the first enlargement module, configured to segment the enlarged first training sample into a plurality of first training blocks;

[0011] A first labeling module, connected to the first segmentation module, is used to label the proportion of each Gleason grade for the first training block to form a first label vector;

[0012] A first training module, connected to the first labeling module, is used to train an initial prostate cancer prediction model using a first training block having a first label vector and store the first training block in a first storage module;

[0013] The first prediction module is connected to the first storage module, and is used to use the trained prostate cancer prediction model to predict the first block to be predicted that is segmented into the pathological section of the prostate to be predicted, and output the grading vector of the first block to be predicted, and obtain the Gleason score result of the pathological section to be predicted according to the grading vector of each first block to be predicted.

[0014] Furthermore, the Gleason grades annotated by the first annotation module include normal glands, Gleason grade 3 cancer, Gleason grade 4 cancer, and Gleason grade 5 cancer.

[0015] Furthermore, the first acquisition module is also used to acquire a pathological section of the prostate to be predicted;

[0016] The first amplification module is also used to amplify the pathological slice to be predicted;

[0017] The first segmentation module is also used to segment the enlarged pathological slice to be predicted to form a plurality of first image blocks to be predicted;

[0018] The first prediction module includes:

[0019] A block grading unit, configured to use the trained prostate cancer prediction model to predict the first block to be predicted and output a grading vector of the first block to be predicted, wherein the grading vector of the first block to be predicted includes a predicted proportion of each Gleason grade in the first block to be predicted;

[0020] A slice grading unit, connected to the image block grading unit, is used to sum the grading vectors of all first image blocks to be predicted, and then divide the sum by the total number of first image blocks to be predicted to obtain a grading vector of the pathological slice to be predicted, wherein the grading vector of the pathological slice includes the proportion of each Gleason grade in the pathological slice;

[0021] The slice scoring unit is connected to the slice grading unit and is used to obtain the Gleason score result of the pathological slice to be predicted based on the proportion analysis of each Gleason grade in the grading vector of the pathological slice.

[0022] Furthermore, the slice scoring unit includes:

[0023] A first judgment subunit is configured to judge whether the proportion of normal glands in the grading vector of the pathological section is a first preset proportion, and output a first judgment result;

[0024] A second judgment subunit is connected to the first judgment subunit, and is used to judge whether the proportion of the second highest Gleason grade exceeds the second preset proportion when the first judgment result is that the proportion of normal glands is not the first preset proportion, so as to obtain a second judgment result;

[0025] The scoring determination subunit is connected to the first judgment subunit and the second judgment subunit respectively, and is used to:

[0026] When the first judgment result is that the proportion of normal glands is the first preset proportion, a normal Gleason score result of prostate cancer is output;

[0027] When the second judgment result is that the proportion of the second highest Gleason grade exceeds the second preset proportion, the Gleason score result is the highest Gleason grade + the second highest Gleason grade;

[0028] When the second judgment result is that the proportion of the second highest Gleason grade does not exceed the second preset proportion, the Gleason score result is the Gleason grade with the highest proportion + the Gleason grade with the highest proportion.

[0029] A method for constructing an adenocarcinoma prediction model based on deep learning, using the aforementioned adenocarcinoma prediction system based on deep learning for scoring prostate cancer, comprising:

[0030] Step A1, obtaining a pathological section of the prostate as a first training sample;

[0031] Step A2, amplifying the first training sample;

[0032] Step A3, segmenting the enlarged first training sample to form a plurality of first training blocks;

[0033] Step A4, annotating the proportion of each Gleason grade for the first training block to form a first label vector;

[0034] Step A5, using the first training block with the first label vector as input, training the initially constructed prostate cancer prediction model, and saving after the training is completed.

[0035] A deep learning-based adenocarcinoma prediction system for breast cancer grading, including:

[0036] A second acquisition module is used to acquire pathological sections of breast as second training samples;

[0037] A second amplification module, connected to the second acquisition module, for amplifying the second training sample;

[0038] A second segmentation module, connected to the second enlargement module, is used to segment the enlarged training sample to form a plurality of second training blocks;

[0039] A second labeling module, connected to the second segmentation module, is used to label the glandular cavity ratio, the atypical nucleus ratio and the number of nuclear division figures for the second training block to form a second label vector;

[0040] a second training module, connected to the second annotation module, for training an initial breast cancer prediction model with a second training block having a second label vector and storing the second training block in a second storage module;

[0041] The second prediction module is connected to the second storage module and is used to use the trained breast cancer prediction model to predict the second block to be predicted formed by segmenting the pathological section to be predicted of the breast, output the prediction vector of the second block to be predicted, and obtain the breast cancer grading result of the pathological section to be predicted based on the prediction vector.

[0042] Furthermore, the second acquisition module is also used to acquire pathological sections to be predicted about the breast;

[0043] The second amplification module is also used to amplify the pathological slice to be predicted;

[0044] The second segmentation module is also used to segment the enlarged pathological slice to be predicted to form a plurality of second image blocks to be predicted;

[0045] The second prediction module includes:

[0046] A block prediction unit, configured to use the trained breast cancer prediction model to predict the second block to be predicted and output a prediction vector of the second block to be predicted, wherein the prediction vector of the second block to be predicted includes the predicted glandular cavity ratio, atypical nucleus ratio, and the number of mitotic figures in the second block to be predicted;

[0047] A slice prediction unit, connected to the image block prediction unit, is used to sum the prediction vectors of all second image blocks to be predicted, and then divide the sum by the total number of second image blocks to be predicted to obtain a prediction vector of the pathological slice to be predicted, wherein the prediction vector of the pathological slice includes the glandular cavity ratio, the atypical nucleus ratio and the number of nuclear division figures;

[0048] A slice scoring unit, connected to the slice prediction unit, is used to score the glandular cavity ratio, the atypical nucleus ratio and the number of nuclear division figures based on the prediction vector of the pathological slice, so as to obtain a scoring vector of the pathological slice;

[0049] Cancer grading unit: connected to the slice scoring unit, used to calculate the total score based on the scoring vector, and obtain the breast cancer grade in the pathological slice to be predicted according to the total score.

[0050] Furthermore, the second labeling module labels the percentage of the glandular cavity, the percentage of the atypical nucleus, and the number of nuclear division images for the second training block, and then converts the percentage of the glandular cavity and the percentage of the atypical nucleus into a percentage with a value between 0 and 1, thereby forming a second label vector including the percentage of the glandular cavity, the percentage of the atypical nucleus, and the number of nuclear division images;

[0051] The Slice Scoring Unit is used to:

[0052] When the glandular cavity proportion is less than 0.1, it is scored as 3 points; when the glandular cavity proportion is not less than 0.1 and not more than 0.75, it is scored as 2 points; when the glandular cavity proportion is more than 0.75, it is scored as 1 point;

[0053] When the proportion of heteromorphic nuclei was less than 0.1, it was scored as 1 point; when the proportion of heteromorphic nuclei was not less than 0.1 and not greater than 0.5, it was scored as 2 points; when the proportion of heteromorphic nuclei was greater than 0.5, it was scored as 3 points;

[0054] When the number of mitotic figures was less than 2, it was scored as 1 point; when the number of mitotic figures was not less than 2 and not greater than 10, it was scored as 2 points; when the number of mitotic figures was greater than 10, it was scored as 3 points.

[0055] Furthermore, the cancer staging unit is used to:

[0056] When the total score is 3-5 points, the output breast cancer grade is grade 1;

[0057] When the total score is 6-7 points, the output breast cancer grade is 2;

[0058] When the total score is 8-9 points, the output breast cancer grade is grade 3.

[0059] A method for constructing an adenocarcinoma prediction model based on deep learning, which is used for grading breast cancer, uses the aforementioned adenocarcinoma prediction system based on deep learning, including:

[0060] Step C1, obtaining a pathological section of the breast as a second training sample;

[0061] Step C2, amplifying the second training sample;

[0062] Step C3, segmenting the enlarged second training sample to form a plurality of second training blocks;

[0063] Step C4, annotating the glandular cavity ratio, the atypical nucleus ratio and the number of nuclear division figures for the second training block to form a second label vector;

[0064] Step C5, using the second training block with the second label vector as input, training the initial breast cancer prediction model, and saving after the training is completed.

[0065] The beneficial technical effect of the present invention is that when training the adenocarcinoma prediction model, the present invention divides the case slices into blocks, annotates the blocks with the percentages of each prediction item to form a label vector to train the model, so that the trained model outputs the percentages of each prediction item of the block in actual prediction, and finally scores or grades the pathological slices based on the percentages of each prediction item of all blocks. Compared with the existing prediction model trained with a single graded / scored block, the accuracy is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1-3 , Figure 5-6 This is a module schematic diagram of an adenocarcinoma prediction system based on deep learning of the present invention;

[0067] Figure 4 and Figure 7 This is a flowchart of the steps of a method for constructing an adenocarcinoma prediction model based on deep learning in the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0070] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0071] See also Figure 1The present invention provides a deep learning-based adenocarcinoma prediction system for scoring prostate cancer, comprising:

[0072] A first acquisition module (1) is used to acquire a pathological section of the prostate as a first training sample;

[0073] A first amplification module (2), connected to the first acquisition module (1), and used for amplifying the first training sample;

[0074] A first segmentation module (3), connected to the first magnification module (2), is used to segment the magnified first training sample to form a plurality of first training blocks;

[0075] A first labeling module (4), connected to the first segmentation module (3), is used to label the proportion of each Gleason grade for the first training block to form a first label vector;

[0076] A first training module (5), connected to the first labeling module (4), is used to train an initial prostate cancer prediction model using a first training image block having a first label vector and store the image in a first storage module;

[0077] The first prediction module (6) is connected to the first storage module (5) and is used to use the trained prostate cancer prediction model to predict the first block to be predicted segmented from the pathological section of the prostate to be predicted, and output the grading vector of the first block to be predicted, and obtain the Gleason score result of the pathological section to be predicted based on the grading vector of each first block to be predicted.

[0078] The pathological slice of the prostate acquired by the first acquisition module (1) is a digital pathological slice formed by scanning the tissue slice of the prostate with a digital pathological scanner. One pathological slice is used as a first training sample, and the number of pathological slices is more than one. Several pathological slices of different symptoms can be acquired as training samples to train the model.

[0079] The first magnifying module (2) magnifies the pathological section at a magnification of 20 times.

[0080] The first segmentation module (3) divides the pathological slice with a magnification of 20 times into first training blocks of 2048x2048 pixels. The divided blocks can be put into a regular graphics card, solving the problem of insufficient graphics memory.

[0081] The first labeling module (4) labels each first training block with the percentage of each Gleason grade, so that the percentage of all Gleason grades in each first training block adds up to 100%, and then converts the percentage between 0-100 to a percentage between 0-1 (i.e., normalizes it to a value between 0-1). For example, the percentage of Gleason grade 4 cancer in the first training block is 10%, and after conversion to a percentage between 0-1, the percentage of Gleason grade 4 cancer in the first training block is 0.1. Afterwards, the first labeling module (4) forms a first label vector with the percentage of each converted Gleason grade in the first training block. For example, the labeled Gleason grades include normal gland, Gleason grade 3 cancer, Gleason grade 4 cancer, and Gleason grade 5 cancer, and the percentages are 0.8, 0.15, 0.05, and 0, respectively, then the first label vector is (0.8, 0.15, 0.05, 0).

[0082] Specifically, the proportion in the present invention refers to the area proportion.

[0083] The initial prostate cancer prediction model is trained using the first training block with the first label vector, and then the trained prostate cancer prediction model is used to predict the segmented blocks of the pathological section to be predicted. The prediction output is also the proportion of each Gleason grade of the block, and then the proportion of all segmented blocks is comprehensively considered to obtain the score of the pathological section regarding prostate cancer.

[0084] It can be seen that when training the adenocarcinoma prediction model, the present invention divides the case slices into blocks, annotates the blocks with the percentage of each prediction item to form a label vector to train the model, so that the trained model outputs the percentage of each prediction item of the block in the actual prediction, and finally scores the pathological slices based on the percentage of each prediction item of all blocks. Compared with the existing prediction model trained with a single scoring annotated block, the accuracy is higher.

[0085] Furthermore, the Gleason grades annotated by the first annotation module (4) include normal glands, Gleason grade 3 cancer, Gleason grade 4 cancer and Gleason grade 5 cancer.

[0086] The Gleason grade of prostate cancer is set to four levels in total, and the first label vector formed is a four-dimensional label vector. Furthermore, the first training module (5) is also used to: before the first training block is input into the prostate cancer prediction model, the first training block is randomly adjusted in terms of brightness, contrast, saturation and hue as data enhancement. This improves the generalization of the model, thereby adapting to various different slice staining encountered in clinical practice.

[0087] Specifically, the prostate cancer prediction model is a convolutional neural network (CNN) model, including:

[0088] A convolutional network, used for extracting feature vectors from the first training block;

[0089] A fully connected network is used to obtain a 4-dimensional vector representing the prostate cancer grade based on the feature vector extracted by the convolutional network;

[0090] The Softmax function converts the values ​​in the output 4-dimensional vector into proportion values ​​of 0-1, and the sum of the values ​​is 1.

[0091] The parameters of the convolutional network and the fully connected network are randomly assigned during initialization.

[0092] When the first training module (5) is forward propagated, the first training block is input into the convolutional network to obtain a feature vector, and then the feature vector is input into the fully connected network to obtain a 4-dimensional vector representing the Gleason grade of prostate cancer. The Softmax function is then used to convert the values ​​in the output 4-dimensional vector into proportion values ​​of 0-1, and the sum of the values ​​is 1, forming a grading vector of the first training block. At this time, the 4 values ​​in the grading vector represent the proportions of normal glands, Gleason grade 3 cancer, Gleason grade 4 cancer, and Gleason grade 5 cancer in the first training block of the model.

[0093] During the back propagation of the first training module (5), the difference between the predicted prostate cancer classification vector and the first label vector in the first training block is calculated by the cross entropy loss function, and the gradient of each parameter is calculated by the loss function. At this time, the ratio in the classification vector is a proportion value of 0-1, and the value in the label vector is also a proportion value of 0-1. Compared with the conventional practice in the prior art of using a one-hot vector as a label to train the prostate model, this method uses the same proportion value label as the model to calculate the loss better.

[0094] The first training module (5) uses the Adam optimizer and a learning rate of 0.001 to update each parameter of the model according to the calculated gradient.

[0095] The first training module (5) repeats the above steps (forward propagation, back propagation and parameter update) for multiple iterative training until the loss of the model converges, thereby obtaining a prostate cancer prediction model.

[0096] See also Figure 2 , further, the first acquisition module (1) is also used to acquire a pathological section of the prostate to be predicted;

[0097] The first amplification module (2) is also used to amplify the pathological slice to be predicted;

[0098] The first segmentation module (3) is also used to segment the amplified pathological slice to be predicted to form a plurality of first image blocks to be predicted;

[0099] The first prediction module (6) comprises:

[0100] A block grading unit (61) is used to use the trained prostate cancer prediction model to predict the first block to be predicted and output a grading vector of the first block to be predicted, wherein the grading vector of the first block to be predicted includes the predicted proportion of each Gleason grade in the first block to be predicted;

[0101] A slice grading unit (62), connected to the image block grading unit (61), is used to sum the grading vectors of all first image blocks to be predicted, and then divide the sum by the total number of first image blocks to be predicted to obtain a grading vector of the pathological slice to be predicted, wherein the grading vector of the pathological slice includes the proportion of each Gleason grade in the pathological slice;

[0102] The slice scoring unit (63) is connected to the slice grading unit (62) and is used to obtain the Gleason score result of the pathological slice to be predicted based on the proportion analysis of each Gleason grade in the grading vector of the pathological slice.

[0103] The magnification and segmentation of the pathological slice to be predicted are processed in the same manner as the magnification and segmentation of the first training sample.

[0104] The image block grading unit (61) predicts the proportion of each Gleason grade in a single image block. After the model training is completed, the pathological sections in the clinical data are inferred, and the first image block to be predicted is formed by enlarging and dicing. The grading vector of the first image block to be predicted is forward propagated, and the grading vector includes the proportion of each Gleason grade in the first image block to be predicted, and is presented in the form of a proportion with a value between 0 and 1. In the previous prostate cancer model, the Argmax function was used to obtain the classification with the highest probability during the prediction process and a single classification was output as the cancer grade of the image block to be predicted. However, the present invention does not need to add the Argmax function, because the data containing the proportion of multiple classifications has been used during training, and the model can also complete the output of the proportion of multiple classifications with high accuracy in the test reasoning.

[0105] The slice grading unit (62) sums the grading vectors of all first to-be-predicted blocks in the pathological slice to be predicted and then divides the sum by the total number of first to-be-predicted blocks to obtain an average, and the result obtained is the grading vector of the pathological slice to be predicted, and the grading vector contains the proportions of normal glands, Gleason grade 3 cancer, Gleason grade 4 cancer and Gleason grade 5 cancer.

[0106] See also Figure 3, further, the slice scoring unit (63) comprises:

[0107] A first judgment subunit (631) judges whether the proportion of normal glands in the grading vector of the pathological section is a first preset proportion, and outputs a first judgment result;

[0108] A second judgment subunit (632), connected to the first judgment subunit (631), is used to judge whether the proportion of the second highest Gleason grade exceeds the second preset proportion when the first judgment result is that the proportion of normal glands is not the first preset proportion, so as to obtain a second judgment result;

[0109] The score determination subunit (633) is connected to the first judgment subunit (631) and the second judgment subunit (632) respectively, and is used to:

[0110] When the first judgment result is that the proportion of normal glands is the first preset proportion, the Gleason score result is output as normal prostate;

[0111] When the second judgment result is that the proportion of the second highest Gleason grade exceeds the second preset proportion, the output Gleason score result is the highest proportion of the Gleason grade + the second highest proportion of the Gleason grade;

[0112] When the second judgment result is that the proportion of the second highest Gleason grade does not exceed the second preset proportion, the output Gleason score result is the Gleason grade with the highest proportion + the Gleason grade with the highest proportion.

[0113] Furthermore, the first preset ratio is 1. Because the first label vector is calculated with a numerical ratio of 0-1, the Gleason grade ratio of the grading vector of the predicted output image block is also between 0-1, and the total ratio is 1, that is, the maximum ratio is 1. If the ratio of normal glands is 1, it indicates that the pathological sections are all normal glands, and there is no Gleason grade 3 cancer, Gleason grade 4 cancer, and Gleason grade 5 cancer.

[0114] Furthermore, the second preset proportion is 0.05. Assuming that Gleason grade 3 cancer has the highest proportion, and Gleason grade 4 cancer has the second highest proportion, if Gleason grade 4 cancer exceeds 0.05 (indicating that the percentage is more than 5%), the Gleason score result is 3+4. If Gleason grade 4 cancer does not exceed 0.05, the Gleason score result is 3+3.

[0115] See also Figure 4The present invention also provides a method for constructing an adenocarcinoma prediction model based on deep learning, using the aforementioned adenocarcinoma prediction system based on deep learning for scoring prostate cancer, characterized in that it includes:

[0116] Step A1, obtaining a pathological section of the prostate as a first training sample;

[0117] Step A2, amplifying the first training sample;

[0118] Step A3, segmenting the enlarged first training sample to form a plurality of first training blocks;

[0119] Step A4, annotating the proportion of each Gleason grade for the first training block to form a first label vector;

[0120] Step A5, using the first training block with the first label vector as input, training the initially constructed prostate cancer prediction model, and saving after the training is completed.

[0121] In step A1, the acquired pathological slice of the prostate is a digital pathological slice formed by scanning the tissue slice of the prostate with a digital pathological scanner. One pathological slice is used as a first training sample, and the number of pathological slices is more than one. Several pathological slices of different symptoms can be obtained as training samples to train the model.

[0122] In step A2, the pathological sections were enlarged at 20-fold magnification.

[0123] In step A3, the pathological sections at 20 times magnification are divided into first training blocks of 2048x2048 pixels. The divided blocks can be put into a regular graphics card, solving the problem of insufficient graphics memory.

[0124] In step A4, the percentage of each Gleason grade is annotated for each first training block, so that the percentage of all Gleason grades in each first training block adds up to 100%, and then the percentage between 0-100 is converted to a percentage between 0-1 (i.e., normalized to a numerical value between 0-1). For example, the percentage of Gleason grade 4 cancer in the first training block is 10%, and after conversion to a percentage between 0-1, the percentage of Gleason grade 4 cancer in the first training block is 0.1. Afterwards, the percentage of each converted Gleason grade in the first training block is formed into a first label vector. For example, the annotated Gleason grades are normal glands, Gleason grade 3 cancer, Gleason grade 4 cancer, and Gleason grade 5 cancer, with percentages of 0.8, 0.15, 0.05, and 0, respectively, then the first label vector is (0.8, 0.15, 0.05, 0).

[0125] Specifically, the proportion in the present invention refers to the area proportion.

[0126] In step A5, the initial prostate cancer prediction model is trained using the first training block with the first label vector, and then the trained prostate cancer prediction model is used to predict the segmented blocks of the pathological slice to be predicted, and the predicted output is also the proportion of each Gleason grade of the block, and then the proportion of the entire segmented block is comprehensively considered to obtain the score of the pathological slice for prostate cancer. When training the adenocarcinoma prediction model, the present invention divides the case slice into blocks, and the blocks are labeled with the percentage of each prediction item to form a label vector to train the model, so that the trained model outputs the percentage of each prediction item of the block in the actual prediction, and finally the percentage of each prediction item of all blocks is comprehensively used to score the pathological slice, which is more accurate than the existing prediction model trained with a single scoring labeled block.

[0127] Further, in step A4, the annotated Gleason grades include normal glands, Gleason grade 3 cancer, Gleason grade 4 cancer, and Gleason grade 5 cancer.

[0128] The Gleason grade of prostate cancer is set to four levels in total, and the first label vector formed is a four-dimensional label vector.

[0129] Furthermore, after step A4 and before step A5, step B1 is further included to randomly adjust the brightness, contrast, saturation and hue of the first training block, so as to improve the generalization of the model and adapt to various different slice staining encountered in clinical practice.

[0130] See also Figure 5 The present invention also provides an adenocarcinoma prediction system based on deep learning, which is used for grading breast cancer, comprising:

[0131] A second acquisition module (7) is used to acquire pathological sections of breast as second training samples;

[0132] A second amplification module (8), connected to the second acquisition module (7), and used for amplifying the second training sample;

[0133] A second segmentation module (9), connected to the second magnification module (8), is used to segment the magnified training sample to form a plurality of second training blocks;

[0134] A second labeling module (10), connected to the second segmentation module (9), is used to label the glandular cavity ratio, the atypical nucleus ratio and the number of nuclear division figures for the second training block to form a second label vector;

[0135] A second training module (11), connected to the second labeling module (10), is used to train an initial breast cancer prediction model with a second training block having a second label vector and store the result in a second storage module;

[0136] The second prediction module (12) is connected to the second storage module (11) and is used to use the trained breast cancer prediction model to predict a second block to be predicted formed by segmenting the pathological section to be predicted of the breast, output a prediction vector of the second block to be predicted, and obtain a breast cancer grading result of the pathological section to be predicted based on the prediction vector.

[0137] The pathological slice of the breast acquired by the second acquisition module (7) is a digital pathological slice formed by scanning the tissue slice of the prostate with a digital pathological scanner.

[0138] The second magnifying module (8) magnifies the pathological section at a magnification of 20 times.

[0139] The second segmentation module (9) divides the pathological sections of the breast at a magnification of 20 times into second training blocks of 2048×2048 pixels. The divided blocks can be put into a regular graphics card, solving the problem of insufficient graphics memory.

[0140] The second labeling module (10) labels the percentage of glandular cavity, the percentage of atypical nuclei and the number of nuclear division images for the second training block, and then converts the percentage of glandular cavity and the percentage of atypical nuclei into percentages with values ​​between 0 and 1, thereby forming a second label vector including the percentage of glandular cavity, the percentage of atypical nuclei and the number of nuclear division images. The number of nuclear division images labeled is a natural number.

[0141] In the present invention, the proportion refers to the area proportion.

[0142] The glandular cavity ratio in the block is the ratio of the area of ​​the gland with the glandular cavity to the total area of ​​all glands in the block, for example, 0.05 (ie, the percentage ratio is 5%).

[0143] The percentage of heteromorphic nuclei in a block refers to the ratio of the area of ​​heteromorphic nuclei to the total area of ​​all cell nuclei in the block, for example 0.1 (ie, the percentage is 10% of the periphery).

[0144] The second label vector is a three-dimensional label vector of the glandular cavity ratio, the atypical nucleus ratio and the number of nuclear division figures. For example, when the glandular cavity ratio, the atypical nucleus ratio and the number of nuclear division figures are 0.05, 0.1 and 3 respectively, the second label vector is (0.05, 0.1, 3).

[0145] The initial breast cancer prediction model is trained using the second training block with the second label vector, and then the trained breast cancer prediction model is used to predict the segmented blocks of the pathological section to be predicted. The prediction output is also the glandular cavity proportion, the atypical nucleus proportion and the number of nuclear division figures of the block, that is, the prediction vector. The proportion of all segmented blocks is comprehensively considered to obtain the score of the pathological section regarding breast cancer.

[0146] When training the adenocarcinoma prediction model, the present invention divides the case slices into blocks, annotates the blocks with the percentage of each prediction item to form a label vector to train the model, so that the trained model outputs the percentage of each prediction item of the block in actual prediction, and finally grades the pathological slices based on the percentage of each prediction item of all blocks. Compared with the existing prediction model trained by a single graded annotated block, the accuracy is higher.

[0147] The second prediction module (12) is also used to randomly adjust the brightness, contrast, saturation and hue of the second training block as data enhancement before the second training block is input into the breast cancer prediction model, so as to improve the generalization of the model and adapt to various different slice staining encountered in clinical practice.

[0148] Specifically, the breast cancer prediction model is a convolutional neural network (CNN) model, including:

[0149] A convolutional network, used for extracting feature vectors from the second training tile;

[0150] A fully connected network is used to obtain a 3D vector representing the prostate cancer grade based on the feature vector extracted by the convolutional network;

[0151] The Softmax function converts the two proportion class values ​​in the output 3D vector into proportion values ​​of 0-1 to obtain the prediction vector of the second training block;

[0152] The parameters of the convolutional network and the fully connected network are randomly assigned during initialization.

[0153] When the second training module (11) propagates forward, the second training block is input into the convolutional network to obtain a feature vector, and then the feature vector is input into the fully connected network to obtain a 3D vector representing breast cancer. The Softmax function is then used to convert the two proportion class values ​​in the output 3D vector into proportion values ​​of 0-1 to form a prediction vector for the second training block. At this time, the three values ​​in the prediction vector represent the glandular cavity proportion, the atypical nucleus proportion and the number of nuclear divisions in the second training block.

[0154] During the back propagation of the second training module (11), the difference between the predicted vector of breast cancer in the second training block and the first label vector is calculated by the cross entropy loss function, and the gradient of each parameter is calculated by the loss function. In contrast to the conventional practice in the prior art of using one-hot vectors as labels to train breast cancer models, this method uses label vectors with the same numerical proportions as the model to calculate the loss better.

[0155] The second training module (11) uses the Adam optimizer and a learning rate of 0.001 to update each parameter of the model according to the calculated gradient.

[0156] The second training module (11) repeats the above steps (forward propagation, back propagation and parameter update) for multiple iterative training until the loss of the model converges, thereby obtaining a breast cancer prediction model.

[0157] See also Figure 6 ,Furthermore, the second acquisition module is also used to acquire the pathological slice to be predicted about the breast;

[0158] The second amplification module (8) is also used to amplify the pathological slice to be predicted;

[0159] The second segmentation module (9) is also used to segment the amplified pathological slice to be predicted to form a plurality of second image blocks to be predicted;

[0160] The second prediction module (12) comprises:

[0161] A block prediction unit (121) is used to use the trained breast cancer prediction model to predict the second block to be predicted and output a prediction vector of the second block to be predicted, wherein the prediction vector of the second block to be predicted includes the predicted glandular cavity ratio, atypical nucleus ratio and the number of nuclear division figures in the second block to be predicted;

[0162] A slice prediction unit (122), connected to the image block prediction unit (121), is used to sum the prediction vectors of all second image blocks to be predicted, and then divide the sum by the total number of second image blocks to be predicted to obtain a prediction vector of the pathological slice to be predicted, wherein the prediction vector of the pathological slice includes the proportion of glandular cavities, the proportion of atypical nuclei, and the number of nuclear division figures;

[0163] A slice scoring unit (123), connected to the slice prediction unit (122), is used to score the glandular cavity ratio, the atypical nucleus ratio and the number of nuclear division images based on the prediction vector of the pathological slice, so as to obtain a scoring vector of the pathological slice;

[0164] Cancer grading unit (124): connected to the slice scoring unit (123), used to calculate a total score based on the scoring vector, and obtain the breast cancer grade in the pathological slice to be predicted according to the total score.

[0165] The magnification and segmentation of the pathological slice to be predicted are processed in the same manner as the magnification and segmentation of the second training sample.

[0166] The block prediction unit (121) realizes the prediction of the glandular cavity proportion, the atypical nucleus proportion and the number of nuclear division images of a single block. After the model training is completed, the pathological sections in the clinical data are inferred, and the second block to be predicted is formed by amplification and dicing. The prediction vector of the second block to be predicted is forward propagated, and the prediction vector includes the glandular cavity proportion, the atypical nucleus proportion and the number of nuclear division images, and the proportion value is in the range of 0-1 (the proportion value multiplied by 100 is the percentage proportion). In the previous breast cancer model, the Argmax function is used to obtain the classification with the maximum probability during the prediction process and output a single classification as the score of the block to be predicted. However, the present invention does not need to add the Argmax function, because the multi-classification data is used during training, and the model can also complete the multi-classification prediction result output with high accuracy in the test reasoning.

[0167] The slice prediction unit (122) sums the prediction vectors of all the second blocks to be predicted in the pathological slice to be predicted and then divides the sum by the total number of the third blocks to be predicted to obtain an average, and the result obtained is the prediction vector of the pathological slice to be predicted. The classification vector contains the glandular cavity ratio, the atypical nucleus ratio and the number of nuclear division figures.

[0168] Furthermore, the second labeling module (10) labels the percentage of glandular cavities, the percentage of atypical nuclei, and the number of nuclear division images for the second training image block, and then converts the percentage of glandular cavities and the percentage of atypical nuclei into percentages with values ​​between 0 and 1, thereby forming a second label vector including the percentage of glandular cavities, the percentage of atypical nuclei, and the number of nuclear division images;

[0169] The slice scoring unit (123) is used to:

[0170] When the glandular cavity proportion is less than 0.1, it is scored as 3 points; when the glandular cavity proportion is not less than 0.1 and not more than 0.75, it is scored as 2 points; when the glandular cavity proportion is more than 0.75, it is scored as 1 point;

[0171] When the proportion of heteromorphic nuclei was less than 0.1, it was scored as 1 point; when the proportion of heteromorphic nuclei was not less than 0.1 and not greater than 0.5, it was scored as 2 points; when the proportion of heteromorphic nuclei was greater than 0.5, it was scored as 3 points;

[0172] When the number of mitotic figures was less than 2, it was scored as 1 point; when the number of mitotic figures was not less than 2 and not greater than 10, it was scored as 2 points; when the number of mitotic figures was greater than 10, it was scored as 3 points.

[0173] Furthermore, the cancer staging unit (124) is used to:

[0174] When the total score is 3-5 points, the output breast cancer grade is grade 1;

[0175] When the total score is 6-7 points, the output breast cancer grade is 2;

[0176] When the total score is 8-9 points, the output breast cancer grade is grade 3.

[0177] See also Figure 7 The present invention also provides a method for constructing an adenocarcinoma prediction model based on deep learning, which is used for grading breast cancer. The method for constructing an adenocarcinoma prediction model based on deep learning includes:

[0178] Step C1, obtaining a pathological section of the breast as a second training sample;

[0179] Step C2, amplifying the second training sample;

[0180] Step C3, segmenting the enlarged second training sample to form a plurality of second training blocks;

[0181] Step C4, annotating the glandular cavity ratio, the atypical nucleus ratio and the number of nuclear division figures for the second training block to form a second label vector;

[0182] Step C5, using the second training block with the second label vector as input, training the initial breast cancer prediction model, and saving after the training is completed.

[0183] In step C1, the acquired breast pathological slice is a digital pathological slice formed by scanning the prostate tissue slice with a digital pathological scanner.

[0184] In step C2, the pathological sections are magnified at 20 times magnification.

[0185] In step C3, the pathological sections of the breast at 20 times magnification are divided into second training blocks of 2048x2048 pixels. The divided blocks can be put into a regular graphics card, solving the problem of insufficient graphics memory.

[0186] In step C4, the percentage of glandular cavity, the percentage of atypical nuclei and the number of nuclear division images are respectively labeled for the second training block, and then the percentage of glandular cavity and the percentage of atypical nuclei are respectively converted into percentages with values ​​between 0 and 1 to form a second label vector including the percentage of glandular cavity, the percentage of atypical nuclei and the number of nuclear division images. The number of nuclear division images labeled is a natural number.

[0187] In the present invention, the proportion refers to the area proportion.

[0188] The glandular cavity ratio in the block is the ratio of the area of ​​the gland with the glandular cavity to the total area of ​​all glands in the block, for example, 0.05 (ie, the percentage ratio is 5%).

[0189] The percentage of heteromorphic nuclei in a block refers to the ratio of the area of ​​heteromorphic nuclei to the total area of ​​all cell nuclei in the block, for example 0.1 (ie, the percentage is 10% of the periphery).

[0190] The second label vector is a three-dimensional label vector of the glandular cavity ratio, the atypical nucleus ratio and the number of nuclear division figures. For example, when the glandular cavity ratio, the atypical nucleus ratio and the number of nuclear division figures are 0.05, 0.1 and 3 respectively, the second label vector is (0.05, 0.1, 3).

[0191] The initial breast cancer prediction model is trained using the second training block with the second label vector, and then the trained breast cancer prediction model is used to predict the segmented blocks of the pathological section to be predicted. The prediction output is also the glandular cavity proportion, the atypical nucleus proportion and the number of nuclear division figures of the block, that is, the prediction vector. The proportion of all segmented blocks is comprehensively considered to obtain the score of the pathological section regarding breast cancer.

[0192] After step C4 and before step C5, the method further includes step B2: before the second training block is input into the breast cancer prediction model, the brightness, contrast, saturation and hue of the second training block are randomly adjusted, so as to improve the generalization of the model and adapt to various different slice staining encountered in clinical practice.

[0193] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A deep learning-based adenocarcinoma prediction system for scoring prostate cancer, characterized in that: include: A first acquisition module, used for acquiring a pathological section of the prostate as a first training sample; a first amplification module, connected to the first acquisition module, and configured to amplify the first training sample; A first segmentation module, connected to the first magnification module, configured to segment the magnified first training sample into a plurality of first training blocks; A first labeling module, connected to the first segmentation module, for labeling the first training block with the proportion of each Gleason grade to form a first label vector; a first training module, connected to the first labeling module, configured to train an initial prostate cancer prediction model using the first training image block having the first label vector and store the image in a first storage module; The first prediction module is connected to the first storage module, and is used to use the trained prostate cancer prediction model to predict the first block to be predicted that is segmented into the pathological section of the prostate to be predicted, and output the grading vector of the first block to be predicted, and obtain the Gleason score result of the pathological section to be predicted according to the grading vector of each of the first blocks to be predicted.

2. A deep learning-based adenocarcinoma prediction system as described in claim 1, characterized in that the Gleason grades annotated by the first annotation module include normal glands, Gleason grade 3 cancer, Gleason grade 4 cancer and Gleason grade 5 cancer.

3. The deep learning-based adenocarcinoma prediction system according to claim 1, characterized in that the first acquisition module is also used to acquire the pathological section of the prostate to be predicted; The first amplification module is also used to amplify the pathological slice to be predicted; The first segmentation module is also used to segment the enlarged pathological slice to be predicted to form a plurality of first image blocks to be predicted; The first prediction module comprises: A block grading unit, configured to use the trained prostate cancer prediction model to predict the first block to be predicted and output a grading vector of the first block to be predicted, wherein the grading vector of the first block to be predicted includes a predicted proportion of each Gleason grade in the first block to be predicted; a slice grading unit, connected to the image block grading unit, configured to sum the grading vectors of all the first image blocks to be predicted, and then divide the sum by the total number of the first image blocks to be predicted to obtain a grading vector of the pathological slice to be predicted, wherein the grading vector of the pathological slice includes a proportion of each Gleason grade in the pathological slice; A slice scoring unit, connected to the slice grading unit, is used to obtain the Gleason score result of the pathological slice to be predicted based on the proportion analysis of each Gleason grade in the grading vector of the pathological slice.

4. The deep learning-based adenocarcinoma prediction system according to claim 3, characterized in that the slice scoring unit comprises: a first judgment subunit, for judging whether the proportion of normal glands in the grading vector of the pathological section is a first preset proportion, and outputting a first judgment result; a second judgment subunit, connected to the first judgment subunit, for judging whether the proportion of the Gleason grade with the second highest proportion exceeds the second preset proportion when the first judgment result is that the proportion of the normal glands is not the first preset proportion, to obtain a second judgment result; The scoring determination subunit is connected to the first judgment subunit and the second judgment subunit respectively, and is used to: When the first judgment result is that the proportion of the normal gland is the first preset proportion, outputting a normal Gleason score result of prostate cancer; When the second judgment result is that the proportion of the second highest Gleason grade exceeds the second preset proportion, the Gleason scoring result is the highest Gleason grade + the second highest Gleason grade; When the second judgment result is that the proportion of the second highest Gleason grade does not exceed the second preset proportion, the Gleason scoring result is the Gleason grade with the highest proportion + the Gleason grade with the highest proportion.

5. A method for constructing an adenocarcinoma prediction model based on deep learning, using an adenocarcinoma prediction system based on deep learning as described in any one of claims 1 to 4 for scoring prostate cancer, characterized in that: include: Step A1, obtaining a pathological section of the prostate as a first training sample; Step A2, amplifying the first training sample; Step A3, dividing the enlarged first training sample into a plurality of first training blocks; Step A4, annotating the proportion of each Gleason grade for the first training block to form a first label vector; Step A5, using the first training block with the first label vector as input, training the initially constructed prostate cancer prediction model, and saving after the training is completed.

6. A deep learning-based adenocarcinoma prediction system for breast cancer grading, characterized in that: include: A second acquisition module is used to acquire pathological sections of breast as second training samples; A second amplification module, connected to the second acquisition module, and configured to amplify the second training sample; A second segmentation module, connected to the second enlargement module, configured to segment the enlarged training sample into a plurality of second training blocks; A second labeling module, connected to the second segmentation module, is used to label the glandular cavity ratio, the atypical nucleus ratio and the number of nuclear division figures for the second training block to form a second label vector; a second training module, connected to the second labeling module, for training an initial breast cancer prediction model with the second training image block having a second label vector and storing the image in a second storage module; A second prediction module is connected to the second storage module, and is used to use the trained breast cancer prediction model to predict a second block to be predicted formed by segmenting the pathological section to be predicted of the breast, output a prediction vector of the second block to be predicted, and obtain a breast cancer grading result of the pathological section to be predicted based on the prediction vector.

7. The deep learning-based adenocarcinoma prediction system according to claim 6, characterized in that: The second acquisition module is also used to acquire the pathological slice to be predicted about the breast; The second amplification module is also used to amplify the pathological section to be predicted; The second segmentation module is also used to segment the enlarged pathological section to be predicted to form a plurality of second image blocks to be predicted; The second prediction module comprises: A block prediction unit, configured to use the trained breast cancer prediction model to predict the second block to be predicted and output a prediction vector of the second block to be predicted, wherein the prediction vector of the second block to be predicted includes a predicted glandular cavity ratio, atypical nucleus ratio, and a number of mitotic figures in the second block to be predicted; a slice prediction unit, connected to the image block prediction unit, configured to sum the prediction vectors of all the second image blocks to be predicted, and then divide the sum by the total number of the second image blocks to be predicted to obtain a prediction vector of the pathological slice to be predicted, wherein the prediction vector of the pathological slice includes a glandular cavity ratio, a heteromorphic nucleus ratio, and a number of mitotic figures; a slice scoring unit, connected to the slice prediction unit, for scoring the glandular cavity proportion, the atypical nucleus proportion and the number of nuclear division figures respectively based on the prediction vector of the pathological slice to obtain a scoring vector of the pathological slice; Cancer grading unit: connected to the slice scoring unit, used to calculate a total score based on the scoring vector, and obtain the breast cancer grade in the pathological slice to be predicted according to the total score.

8. The deep learning-based adenocarcinoma prediction system according to claim 7, characterized in that: The second labeling module labels the percentage of glandular cavity, the percentage of atypical nuclei and the number of nuclear division images for the second training block, and then converts the percentage of glandular cavity and the percentage of atypical nuclei into percentages with values ​​between 0 and 1, to form a second label vector including the percentage of glandular cavity, the percentage of atypical nuclei and the number of nuclear division images; The Slice Scoring Unit is used to: When the glandular cavity accounts for less than 0.1%, it is scored as 3 points; when the glandular cavity accounts for not less than 0.1 and not more than 0.75%, it is scored as 2 points; when the glandular cavity accounts for more than 0.75%, it is scored as 1 point; When the proportion of heteromorphic nuclei is less than 0.1, it is scored as 1 point; when the proportion of heteromorphic nuclei is not less than 0.1 and not more than 0.5, it is scored as 2 points; when the proportion of heteromorphic nuclei is more than 0.5, it is scored as 3 points; When the number of nuclear division figures is less than 2, it is scored as 1 point; when the number of nuclear division figures is not less than 2 and not more than 10, it is scored as 2 points; when the number of nuclear division figures is more than 10, it is scored as 3 points.

9. The deep learning-based adenocarcinoma prediction system according to claim 7, characterized in that: The cancer staging unit is used to: When the total score is 3-5 points, the output breast cancer grade is grade 1; When the total score is 6-7 points, the output breast cancer grade is grade 2; When the total score is 8-9 points, the output breast cancer grade is grade 3.

10. A method for constructing an adenocarcinoma prediction model based on deep learning, used for grading breast cancer, characterized in that: Using a deep learning-based adenocarcinoma prediction system as described in any one of claims 6 to 9 comprises: Step C1, obtaining a pathological section of the breast as a second training sample; Step C2, amplifying the second training sample; Step C3, dividing the enlarged second training sample into a plurality of second training blocks; Step C4, annotating the second training block by the percentage of glandular cavities, the percentage of atypical nuclei, and the number of nuclear division figures to form a second label vector; Step C5: using the second training block with the second label vector as input, training an initial breast cancer prediction model, and saving the model after the training is completed.

Images