Lung cancer pathological tissue typing system based on deep semantic segmentation network

Abstract

The application discloses a lung cancer pathological tissue typing system based on a deep semantic segmentation network, and comprises a lung cancer recognition model and a lung cancer typing recognition model; image prediction processing is performed on an input pathological tissue slice image according to the lung cancer recognition model, a first probability of each pixel point in the slice image being cancer is predicted based on deep semantic segmentation network technology, and pixel points with the first probability greater than or equal to a first probability threshold are set as cancerous regions; all cancerous region images are input into the lung cancer typing recognition model, a second probability of each pixel point in the cancerous region image being each cancer type is determined based on deep semantic segmentation network technology, a cancer type corresponding to a pathological tissue slice image and a prediction probability of the cancer type are determined according to a statistical value of the second probability. Through prediction of the cancerous region and the cancer type to which each pixel point in the cancerous region belongs, the prediction probability of the corresponding cancer type is statistically counted, and the accuracy of cancer type classification is improved.

Description

Technical Field

[0001] This invention relates to, in particular to, a lung cancer pathological tissue typing system based on a deep semantic segmentation network. Background Technology

[0002] In 2018, the World Health Organization's Global Cancer Report stated that lung cancer has become the leading cause of cancer-related morbidity and mortality worldwide. Precision treatment for lung cancer prioritizes accurate diagnosis and clear pathological classification, especially for the most common types: adenocarcinoma, squamous cell carcinoma, and small cell carcinoma. With the increasing number of patients, the workload of pathology departments is constantly increasing, placing a heavier burden on doctors, making the introduction of artificial intelligence-assisted diagnosis extremely urgent.

[0003] Therefore, in recent years, artificial intelligence has begun to be applied in the medical field. In pathology, deep learning has achieved certain results in the artificial intelligence diagnosis of organs such as the breast, gastrointestinal tract, lymph nodes, and prostate. In the field of artificial intelligence pathology research on the lungs, Professor Coudray used the Inception v3 image classification model only to test non-small cell lung cancer, and was able to complete the identification of lung cancer and the diagnosis of both adenocarcinoma and squamous cell carcinoma. Because Inception v3 can only complete image patch-level classification tasks, its model cannot accurately delineate tumor areas and obtain pixel-level recognition results. Therefore, there is an urgent need for a recognition model that can achieve pixel-level accuracy to improve the accuracy of lung cancer area recognition. Summary of the Invention

[0004] This invention provides a lung cancer pathological tissue typing system based on deep semantic segmentation networks to solve the above-mentioned problems in the prior art.

[0005] This invention provides a lung cancer pathological tissue typing system based on a deep semantic segmentation network, including: a lung cancer identification model and a lung cancer typing identification model;

[0006] The lung cancer identification model performs image prediction processing on the input pathological tissue slice image, predicts the first probability that each pixel in the slice image is cancer based on deep semantic segmentation network technology, and sets the pixels with the first probability greater than or equal to the first probability threshold as cancerous areas.

[0007] All cancerous region images are input into the lung cancer subtype identification model. Based on deep semantic segmentation network technology, the second probability of each pixel in the cancerous region image for each cancer subtype is determined. Based on the statistical value of the second probability, the cancer subtype corresponding to a pathological tissue slice image and the predicted probability of that cancer subtype are determined.

[0008] Preferably, determining the cancer subtype corresponding to a pathological tissue slice image and the predicted probability of that cancer subtype based on the statistical value of the second probability includes:

[0009] Based on the cancer subtype, several categories are divided. Each category includes the cancer subtype type and the second probability of a corresponding pixel belonging to that cancer subtype. The second probability values ​​in each category are sorted, and the number of pixels with the highest second probability is counted, or the number of pixels with a second probability greater than a set second probability threshold is counted. The counted pixels are set as the subtype region. All pixels in the subtype region are classified according to their respective cancer subtypes to form subclasses. The average value of the second probability corresponding to the pixels in each subclass is calculated, and this average value is used as the predicted probability of the corresponding cancer subtype in each subclass.

[0010] Preferably, the lung cancer subtype identification model includes: dividing randomly selected tissue slice images as samples into a training set, a validation set, and a test set;

[0011] The lung cancer classification and identification model is trained by classifying samples using the training set. The trained lung cancer classification and identification model is then iteratively evaluated using the validation set to select the optimal lung cancer subtype identification model. Finally, the optimal lung cancer subtype identification model is evaluated using evaluation metrics on the test set.

[0012] The training set includes: segmenting tissue slice images in the training set into a grid with pixels as the unit based on a deep semantic segmentation network model; and training the classification of samples based on the annotations by labeling the cancer subtypes corresponding to each pixel.

[0013] The validation set includes: iteratively training the model using the errors of sample data in the validation set to obtain a model that fits the dataset reasonably, thus forming the optimal cancer subtyping identification model.

[0014] Preferably, determining the second probability of each pixel in the cancerous region image representing each cancer subtype based on deep semantic segmentation network technology includes:

[0015] The lung cancer subtype identification model records a standard model for each type of cancer. The standard model is a set of images representing the cell and tissue morphology of the corresponding cancer subtype, composed of multiple standard images. The cell and tissue corresponding to each pixel in the cancerous region image is compared with the standard model, and the similarity probability is calculated based on the similarity. The similarity probability is set as the second probability of the corresponding cancer subtype.

[0016] Preferably, in the lung cancer subtype identification model, if the cell tissue corresponding to a certain pixel belongs to two or more cancer subtypes, the pixel is compared with the corresponding standard model for each cancer subtype. If the similarity probability with the standard model of one of the cancer subtypes exceeds a set threshold, then the pixel belongs to that cancer subtype. If the similarity probability with the standard models of two or more cancer subtypes exceeds the set threshold, then the pixel corresponds to multiple cancer subtypes, and the similarity probability corresponding to each cancer subtype is the second probability of that cancer subtype.

[0017] Preferably, when the cell tissue corresponding to a pixel belongs to two or more cancer subtypes, the cancer subtype with the first and second highest similarity probabilities is taken as the primary classification; the region with the highest similarity probability is set as the foreground, and the region with the second highest similarity probability is set as the background.

[0018] All regions in the foreground are formed into foreground regions, and all regions in the background are formed into background regions. When using the foreground regions as the lung cancer subtyping identification model for probability prediction, the evaluation index of the lung cancer subtyping identification model is calculated as the first index. When using the background regions as the lung cancer subtyping identification model for probability prediction, the evaluation index of the lung cancer subtyping identification model is calculated as the second index.

[0019] The first indicator is compared with the second indicator. If the first indicator is better than the second indicator, the cancer subtype corresponding to the foreground region and the corresponding similarity probability are selected as feedback data and fed back to the lung cancer subtype identification model. If the second indicator is better than the first indicator, the cancer subtype corresponding to the background region and the corresponding similarity probability are selected as feedback data and fed back to the lung cancer subtype identification model.

[0020] Preferably, the cancer subtype corresponding to each pixel is labeled, including:

[0021] In the first self-service annotation unit, relevant professionals perform manual annotation, and machine learning is used based on the manual annotation to form a self-service annotation model. The cancer subtype corresponding to each pixel is annotated according to the self-service annotation model.

[0022] The second self-service annotation unit uses a self-service annotation model for initial annotation. Relevant professionals then review and correct any inaccurate areas annotated by the self-service annotation model.

[0023] Preferably, the cancer subtypes include: adenocarcinoma, squamous cell carcinoma, and small cell carcinoma;

[0024] The sample types are surgical resection samples, biopsy samples, and cytological samples.

[0025] Preferably, the deep semantic segmentation network technology uses DeepLab v3, and the feature extraction network equipped with it is ResNet-50, which is used to output pixel-level recognition results.

[0026] Preferably, the evaluation metrics include: AUC value, sensitivity, and specificity;

[0027] The AUC value is the area under the ROC curve. Compared with the prior art, the present invention has the following advantages:

[0028] This invention provides a lung cancer pathological tissue typing system based on deep semantic segmentation networks. It sets up a lung cancer identification model and a lung cancer typing identification model. Based on the lung cancer identification model, it performs image prediction processing on input pathological tissue slice images, predicting a first probability that each pixel in the slice image is cancerous using deep semantic segmentation network technology. Pixels with a first probability greater than or equal to a first probability threshold are designated as cancerous regions. All cancerous region images are input into the lung cancer typing identification model. Based on deep semantic segmentation network technology, it determines a second probability that each pixel in the cancerous region image belongs to a specific cancer type. Based on the statistical value of the second probability, it determines the cancer type corresponding to a pathological tissue slice image and the predicted probability of that cancer type. By utilizing deep semantic segmentation network technology to predict cancerous regions and the cancer type to which each pixel in the cancerous region belongs, and by statistically analyzing the predicted probability of the corresponding cancer type, this solution can achieve pixel-level cancer typing, improving the accuracy of cancer typing classification.

[0029] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0030] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0031] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0032] Figure 1 This is a schematic diagram of the lung cancer pathological tissue typing system based on a deep semantic segmentation network in an embodiment of the present invention;

[0033] Figure 2 This is the ROC curve of the lung cancer identification model in this embodiment of the invention;

[0034] Figure 3This is the ROC curve of the lung cancer subtyping identification model in this embodiment of the invention. Detailed Implementation

[0035] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0036] This invention provides a lung cancer pathological tissue typing system based on a deep semantic segmentation network. Please refer to [link / reference]. Figure 1 This lung cancer pathological tissue typing system based on a deep semantic segmentation network includes:

[0037] Lung cancer identification model, lung cancer subtype identification model;

[0038] The lung cancer identification model performs image prediction processing on the input pathological tissue slice image, predicts the first probability that each pixel in the slice image is cancer based on deep semantic segmentation network technology, and sets the pixels with the first probability greater than or equal to the first probability threshold as cancerous areas.

[0039] All cancerous region images are input into the lung cancer subtype identification model. Based on deep semantic segmentation network technology, the second probability of each pixel in the cancerous region image for each cancer subtype is determined. Based on the statistical value of the second probability, the cancer subtype corresponding to a pathological tissue slice image and the predicted probability of that cancer subtype are determined.

[0040] The working principle of the above technical solution is as follows: The solution adopted in this embodiment is to set up a lung cancer identification model and a lung cancer subtype identification model; according to the lung cancer identification model, the input pathological tissue slice image is subjected to image prediction processing, and the first probability of each pixel in the slice image being cancer is predicted based on deep semantic segmentation network technology. Pixels with the first probability greater than or equal to the first probability threshold are set as cancerous regions; all cancerous region images are input into the lung cancer subtype identification model, and the second probability of each pixel in the cancerous region image being each cancer subtype is determined based on deep semantic segmentation network technology. The cancer subtype corresponding to a pathological tissue slice image and the predicted probability of the corresponding cancer subtype are determined based on the statistical value of the second probability.

[0041] Two deep learning models, namely a lung cancer identification model and a lung cancer subtyping identification model, are cascaded to form a lung cancer subtyping prediction system. During the prediction process, after extracting effective tissue from the pathological images, the lung cancer identification model first performs binary classification to obtain cancerous and non-cancer regions, distinguishing between "cancer" and "non-cancer" to predict cancerous areas. Subsequently, all cancerous region images are input into the lung cancer subtyping identification model for multi-classification processing, providing specific subtyping predictions, such as "adenocarcinoma," "squamous cell carcinoma," and "small cell carcinoma," among other subtypes.

[0042] The beneficial effects of the above technical solution are as follows: The solution provided in this embodiment uses deep semantic segmentation network technology to predict cancerous regions and the cancer subtypes to which each pixel in the cancerous region belongs, and statistically calculates the prediction probability of the corresponding cancer subtypes. Therefore, the solution provided in this embodiment can achieve pixel-level cancer subtyping and improve the accuracy of cancer subtyping classification.

[0043] In another embodiment, determining the cancer subtype corresponding to a pathological tissue slice image and the predicted probability corresponding to that cancer subtype based on the statistical value of the second probability includes:

[0044] Based on the cancer subtype, several categories are divided. Each category includes the cancer subtype type and the second probability of a corresponding pixel belonging to that cancer subtype. The second probability values ​​in each category are sorted, and the number of pixels with the highest second probability is counted, or the number of pixels with a second probability greater than a set second probability threshold is counted. The counted pixels are set as the subtype region. All pixels in the subtype region are classified according to their respective cancer subtypes to form subclasses. The average value of the second probability corresponding to the pixels in each subclass is calculated, and this average value is used as the predicted probability of the corresponding cancer subtype in each subclass.

[0045] The working principle of the above technical solution is as follows: The solution adopted in this embodiment is to determine the cancer subtype corresponding to a pathological tissue slice image and the predicted probability of the corresponding cancer subtype based on the statistical value of the second probability. This includes: dividing the cancer subtype into several categories, each category including the type of cancer subtype and the second probability of the corresponding pixel belonging to the cancer subtype; sorting the second probability values ​​in each category, and counting the number of pixels with the highest second probability, or counting the pixels with the second probability greater than a set second probability threshold, and setting the counted pixels as the subtype region; classifying all pixels in the subtype region according to their respective cancer subtypes to form subclasses, and counting the average value of the second probability corresponding to the pixels in each subclass, and using the average value as the predicted probability of the corresponding cancer subtype in each subclass.

[0046] The predicted probabilities are obtained statistically. Specifically, after determining the probability of each pixel belonging to each cancer subtype, for the prediction of each disease category, the average of the top 1000 pixels with the highest probabilities in the entire slice can be taken as the predicted probability of the corresponding category for the entire slice. Alternatively, the average of all pixels whose probabilities in the entire slice exceed a set second probability threshold can be taken as the predicted probability of the corresponding category for the entire slice.

[0047] The beneficial effects of the above technical solution are as follows: the solution provided in this embodiment can not only accurately locate the type of cancer and the predicted probability corresponding to each cancer subtype, but also determine the regional location of each cancer subtype, achieving the effect of accurate subtyping and localization.

[0048] In another embodiment, the lung cancer subtyping identification model includes: dividing randomly selected tissue slice images as samples into a training set, a validation set, and a test set;

[0049] The lung cancer classification and identification model is trained by classifying samples using the training set. The trained lung cancer classification and identification model is then iteratively evaluated using the validation set to select the optimal lung cancer subtype identification model. Finally, the optimal lung cancer subtype identification model is evaluated using evaluation metrics on the test set.

[0050] The training set includes: segmenting tissue slice images in the training set into a grid with pixels as the unit based on a deep semantic segmentation network model; and training the classification of samples based on the annotations by labeling the cancer subtypes corresponding to each pixel.

[0051] The validation set includes: iteratively training the model using the errors of sample data in the validation set to obtain a model that fits the dataset reasonably, thus forming the optimal cancer subtyping identification model.

[0052] The working principle of the above technical solution is as follows: The solution adopted in this embodiment is that the lung cancer subtyping and identification model includes: randomly selected tissue slice images as samples and divided into training set, validation set and test set;

[0053] The training set is used to classify and train samples to develop a lung cancer classification and identification model. The trained model is then iteratively evaluated using a validation set to select the optimal lung cancer subtype identification model. Finally, the optimal model is evaluated using evaluation metrics on a test set. The training set comprises: segmenting tissue slice images into a pixel-based grid using a deep semantic segmentation network model; labeling each pixel with its corresponding cancer subtype; and classifying and training samples based on the labels. The validation set comprises: iteratively training the model using the errors in the validation set's sample data to obtain a model that fits the dataset reasonably well, thus forming the optimal cancer subtype identification model.

[0054] It should be noted that 1115 lung cancer tissue sections with confirmed immunohistochemical diagnosis, preserved in the pathology department of a hospital from 2016 to 2019, were randomly selected. The sample types included surgical resection samples, biopsy samples, and cytological samples. The tissue types included 392 adenocarcinoma samples, 384 squamous cell carcinoma samples, and 339 small cell carcinoma samples. An additional 229 normal lung tissue sections were selected. These were scanned to form digital slides, which were then randomly divided into a training set of 934 slides (all surgical resection samples, including normal tissue sections). The training set consisted of 105 images of lung cancer, 231 images of adenocarcinoma, 311 images of squamous cell carcinoma, and 287 images of small cell carcinoma; 150 images for validation (all surgical resection samples, including 50 normal tissue sections, 24 adenocarcinoma sections, 38 squamous cell carcinoma sections, and 38 small cell carcinoma sections); and 260 images for testing (surgical resection samples: 74 normal lung tissue sections, 86 adenocarcinoma sections, 29 squamous cell carcinoma sections, and 12 small cell carcinoma sections; biopsy samples: 50 adenocarcinoma sections, 6 squamous cell carcinoma sections, and 2 small cell carcinoma sections; cytology samples: 1 adenocarcinoma section). The training set was labeled (labels included adenocarcinoma, squamous cell carcinoma, small cell carcinoma, non-tumor, poor quality, and ignore) and reviewed by attending physicians and above. A pixel-level lung cancer recognition model and a lung cancer subtype recognition model were built using a ResNet-50-based DeepLab v3 image segmentation model. The optimal model was selected for iteration on the validation set, and then the model's performance was evaluated on the test set.

[0055] The following table shows the data distribution of the training, validation, and test sets: Unit (sheets)

[0056]

[0057]

[0058] The following table shows the test results of the lung cancer identification model:

[0059]

[0060] The following table shows the test results of the lung cancer subtyping identification model:

[0061]

[0062]

[0063] In summary, the training set in this embodiment contained 934 slides, all surgically resected samples, covering adenocarcinoma, squamous cell carcinoma, small cell carcinoma, and normal lung tissue. The test set included surgically resected samples, biopsy samples, and cytological samples. To comprehensively test the performance of the deep learning model, the test samples covered multiple subtypes. All slides were digitized using a scanner at 400x magnification. Pathologists used an annotation system based on iPad and Apple Pencil to accurately annotate all cancer areas in the training set (labels included adenocarcinoma, squamous cell carcinoma, small cell carcinoma, non-tumor, poor quality, and ignore), and delineated typical subtype regions within the cancer area. Each slide underwent three steps: initial review by a physician, secondary review by a physician, and final review by the chief physician, to form the training data for the deep learning model. During data preprocessing, all regions labeled "poor quality" and "ignore" were discarded to prevent interference with model learning.

[0064] The deep learning-based lung cancer subtyping model can provide a more reliable subtyping diagnosis on the basis of lung cancer identification (AUC value = 0.970, sensitivity 94.1%, specificity 94.6%). The sensitivity / specificity of squamous cell carcinoma, adenocarcinoma, and small cell carcinoma on the test set are as follows: Overall test set: 88.6% / 81.8%, 89.7% / 83.1%, 83.3% / 94.3%; Surgically resected samples: 87.6% / 79.7%, 89.5% / 80.9%, 82.4% / 75.0%; Biopsy samples: 92.9% / 85.8%, 91.7% / 86.2%, 100% / 100%; Cytological samples: accurate histological subtyping and localization.

[0065] In another embodiment, determining the second probability of each pixel in the cancerous region image representing a specific cancer subtype based on deep semantic segmentation network technology includes:

[0066] The lung cancer subtype identification model records a standard model for each type of cancer. The standard model is a set of images representing the cell and tissue morphology of the corresponding cancer subtype, composed of multiple standard images. The cell and tissue corresponding to each pixel in the cancerous region image is compared with the standard model, and the similarity probability is calculated based on the similarity. The similarity probability is set as the second probability of the corresponding cancer subtype.

[0067] The working principle of the above technical solution is as follows: The solution adopted in this embodiment is based on deep semantic segmentation network technology to determine the second probability of each pixel in the cancerous region image as each cancer subtype, including: recording a standard model for each cancer subtype in the lung cancer subtype identification model, wherein the standard model is a set of images representing the cell tissue morphology of the corresponding cancer subtype composed of multiple standard images; comparing the cell tissue corresponding to each pixel in the cancerous region image with the standard model, calculating the similarity probability based on the similarity, and setting the similarity probability as the second probability of the corresponding cancer subtype.

[0068] It should be noted that in the supervised learning of the deep semantic segmentation network technology in this embodiment, the following loss function is used to evaluate and urge the network to obtain recognition results.

[0069]

[0070]

[0071]

[0072] L=L1+α1L2+α2L3

[0073] Where m represents the number of categories, n represents the number of pixels in the predicted segmentation map for each category, and y truth y represents the actual value of each pixel. pred G represents the predicted value for each pixel, L1 represents the first loss function, L2 represents the second loss function, and G represents the predicted value for each pixel. truth G represents the feature values ​​of a standard image. pred θ represents the feature value of the predicted graph, θ represents a very small positive value, and the value is 0.0001 to prevent the denominator from being zero. L3 represents the third loss function, α1 represents the first weight parameter, which can be set to 1, and α2 represents the second weight parameter, which can be set to 1.

[0074] While establishing a loss function using the true and predicted values ​​of each pixel, it also considers ignoring the background class that is correctly predicted but has a large proportion of pixels in the overall image, thus correctly predicting lesion areas and improving prediction accuracy.

[0075] The beneficial effects of the above technical solution are as follows: the solution provided in this embodiment is a solution for obtaining the second probability, and the second probability is accurately calculated based on the similarity probability method through the solution of this embodiment.

[0076] In another embodiment, in the lung cancer subtype identification model, if the cell tissue corresponding to a certain pixel belongs to two or more cancer subtypes, the pixel is compared with the corresponding standard model for each cancer subtype. If the similarity probability with the standard model of one of the cancer subtypes exceeds a set threshold, then the pixel belongs to that cancer subtype. If the similarity probability with the standard models of two or more cancer subtypes exceeds the set threshold, then the pixel corresponds to multiple cancer subtypes, and the similarity probability corresponding to each cancer subtype is the second probability of that cancer subtype.

[0077] The working principle of the above technical solution is as follows: In the lung cancer subtype identification model, if the cell tissue corresponding to a certain pixel belongs to two or more cancer subtypes, the pixel is compared with the corresponding standard model for each cancer subtype. If the similarity probability with the standard model of one of the cancer subtypes exceeds a set threshold, then the pixel belongs to that cancer subtype. If the similarity probability with the standard models of two or more cancer subtypes exceeds the set threshold, then the pixel corresponds to multiple cancer subtypes, and the similarity probability corresponding to each cancer subtype is the second probability of that cancer subtype.

[0078] The beneficial effects of the above technical solution are as follows: When the cell tissue corresponding to a certain pixel belongs to two or more cancer subtypes, the solution provided in this embodiment can be used to correspond all cancer subtypes that meet the conditions to the same pixel according to the set rules, so as to achieve the identification result of one pixel corresponding to multiple cancer subtypes, and the probability of each cancer subtype can also be calculated.

[0079] In another embodiment, when the cell tissue corresponding to a pixel belongs to two or more cancer subtypes, the cancer subtype with the first and second most similar probabilities is taken as the primary classification; the region with the highest similarity probability is set as the foreground, and the region with the second most similarity probability is set as the background.

[0080] All regions in the foreground are formed into foreground regions, and all regions in the background are formed into background regions. When using the foreground regions as the lung cancer subtyping identification model for probability prediction, the evaluation index of the lung cancer subtyping identification model is calculated as the first index. When using the background regions as the lung cancer subtyping identification model for probability prediction, the evaluation index of the lung cancer subtyping identification model is calculated as the second index.

[0081] The first indicator is compared with the second indicator. If the first indicator is better than the second indicator, the cancer subtype corresponding to the foreground region and the corresponding similarity probability are selected as feedback data and fed back to the lung cancer subtype identification model. If the second indicator is better than the first indicator, the cancer subtype corresponding to the background region and the corresponding similarity probability are selected as feedback data and fed back to the lung cancer subtype identification model.

[0082] The working principle of the above technical solution is as follows: In this embodiment, when the cell tissue corresponding to a pixel belongs to two or more cancer subtypes, the cancer subtypes with the first and second highest similarity probabilities are taken as the primary classifications; the region with the highest similarity probability is set as the foreground, and the region with the second highest similarity probability is set as the background; all regions in the foreground are formed into foreground regions, and all regions in the background are formed into background regions; when using the foreground region as the lung cancer subtype identification model for probability prediction, the evaluation index of the lung cancer subtype identification model is calculated as the first index, and when using the background region as the lung cancer subtype identification model for probability prediction, the evaluation index of the lung cancer subtype identification model is calculated as the second index; the first index and the second index are compared, and if the first index is better than the second index, the cancer subtype corresponding to the foreground region and the corresponding similarity probability are selected as feedback data and fed back to the lung cancer subtype identification model; if the second index is better than the first index, the cancer subtype corresponding to the background region and the corresponding similarity probability are selected as feedback data and fed back to the lung cancer subtype identification model.

[0083] The beneficial effects of the above technical solution are as follows: by setting a foreground region and a background region, the solution provided in this embodiment feeds data from two directions into the lung cancer subtyping identification model, thereby improving the identification accuracy of the lung cancer subtyping identification model.

[0084] In another embodiment, the cancer subtype corresponding to each pixel is labeled, including:

[0085] In the first self-service annotation unit, relevant professionals perform manual annotation, and machine learning is used based on the manual annotation to form a self-service annotation model. The cancer subtype corresponding to each pixel is annotated according to the self-service annotation model.

[0086] The second self-service annotation unit uses a self-service annotation model for initial annotation. Relevant professionals then review and correct any inaccurate areas annotated by the self-service annotation model.

[0087] The working principle of the above technical solution is as follows: The solution adopted in this embodiment is to annotate the cancer subtype corresponding to each pixel, including: a first self-service annotation unit, in which relevant professionals perform manual annotation, and machine learning is performed based on the manual annotation to form a self-service annotation model, and the cancer subtype corresponding to each pixel is annotated according to the self-service annotation model; a second self-service annotation unit, in which preliminary annotation is performed through the self-service annotation model, and relevant professionals review the annotation based on the preliminary annotation content and correct the inaccurate areas of the self-service annotation model.

[0088] The beneficial effects of the above technical solution are as follows: the solution provided in this embodiment can be used for self-labeling, or self-labeling followed by review and correction by professional pathologists, to ensure the accuracy of the labeled area and form a training set sample with accurate data.

[0089] In another embodiment, the cancer subtypes include: adenocarcinoma, squamous cell carcinoma, and small cell carcinoma;

[0090] The sample types are surgical resection samples, biopsy samples, and cytological samples.

[0091] In another embodiment, the deep semantic segmentation network technology uses DeepLab v3, and the feature extraction network is ResNet-50, which is used to output pixel-level recognition results.

[0092] All deep learning models used DeepLab v3, equipped with a ResNet-50 feature extraction network, capable of outputting pixel-level recognition results. The server ran Ubuntu, and the deep learning framework used was TensorFlow. Training was data-parallel, utilizing four GPUs. The optimization hyperparameters were set as follows: ADAM optimizer, learning rate of 0.0005, batch size of 128 (i.e., 32×4 GPUs), and 400,000 iterations.

[0093] In another embodiment, the evaluation metrics include: AUC value, sensitivity, and specificity;

[0094] The AUC value is the area under the ROC curve.

[0095] The working principle of the above technical solution is as follows: The solution adopted in this embodiment is to calculate sensitivity and specificity using the following method:

[0096] Sensitivity = TP / (TP + FN)

[0097] Specificity = TN / (TN+FP)

[0098] TP (true positive), FP (false positive), TN (true negative), and FN (false negative) represent true positive, false positive, true negative, and false negative, respectively. The optimal model performance in the ROC curve represents the model's effectiveness in the real world.

[0099] This application utilizes nearly 1000 training images to establish an intelligent identification system for lung pathological tissue typing based on a deep semantic segmentation model. Data shows that the deep learning model achieves an AUC value of 0.970 for cancer area identification, demonstrating good recognition performance in both small cell lung cancer and non-small cell lung cancer biopsy and surgical resection samples. The typing model exhibits a sensitivity of over 82.4% and a specificity exceeding 75.0% for identifying adenocarcinoma, squamous cell carcinoma, and small cell carcinoma.

[0100] Specifically, such as Figure 2 and Figure 3 The figures show the ROC curves of the lung cancer identification model and the lung cancer subtyping identification model, respectively. According to the figures, the AUC value of the lung cancer subtyping identification model reaches 0.970, with a sensitivity of 94.1% and a specificity of 94.6%. Regarding lung cancer subtyping, the AUC value for adenocarcinoma is 0.877, with a sensitivity of 87.6% and a specificity of 79.7%; the AUC value for squamous cell carcinoma is 0.897, with a sensitivity of 88.6% and a specificity of 81.8%; and the AUC value for small cell carcinoma is 0.937, with a sensitivity of 92.9% and a specificity of 85.8%. For both surgically resected and biopsy samples, the sensitivity for all three different types of lung cancer is above 82.4%, and the specificity is above 75.0%.

[0101] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A lung cancer pathological tissue typing system based on deep semantic segmentation networks, characterized in that, include: Lung cancer identification model, lung cancer subtype identification model; The lung cancer identification model performs image prediction processing on the input pathological tissue slice image, predicts the first probability that each pixel in the slice image is cancer based on deep semantic segmentation network technology, and sets the pixels with the first probability greater than or equal to the first probability threshold as cancerous areas. All cancerous region images are input into the lung cancer subtype identification model. Based on deep semantic segmentation network technology, the second probability of each pixel in the cancerous region image for each cancer subtype is determined. Based on the statistical value of the second probability, the cancer subtype corresponding to a pathological tissue slice image and the predicted probability of the corresponding cancer subtype are determined. The method for determining the second probability of each pixel in a cancerous region image for each cancer subtype based on deep semantic segmentation network technology includes: recording a standard model for each cancer subtype in the lung cancer subtype identification model, wherein the standard model is a set of images representing the cell tissue morphology of the corresponding cancer subtype, composed of multiple standard images; comparing the cell tissue corresponding to each pixel in the cancerous region image with the standard model, calculating the similarity probability based on the similarity, and setting the similarity probability as the second probability of the corresponding cancer subtype; in the lung cancer subtype identification model, if the cell tissue corresponding to a certain pixel belongs to two or more cancer subtypes, comparing it with the similarity of each cancer subtype with the standard model. For comparison with the corresponding standard models, if the similarity probability with one of the standard models of cancer subtypes exceeds a set threshold, then the pixel belongs to that cancer subtype. If the similarity probability with the standard models of two or more cancer subtypes exceeds the set threshold, then the pixel corresponds to multiple cancer subtypes, and the similarity probability corresponding to each cancer subtype is the second probability of that cancer subtype. When the cell tissue corresponding to a pixel belongs to two or more cancer subtypes, the cancer subtype with the first and second most similar probabilities is taken as the primary classification. The region with the highest similarity probability is set as the foreground, and the region with the second highest similarity probability is set as the background. All regions in the foreground are formed into foreground regions, and all regions in the background are formed into background regions. When using the foreground regions as the lung cancer subtyping identification model for probability prediction, the evaluation index of the lung cancer subtyping identification model is calculated as the first index. When using the background regions as the lung cancer subtyping identification model for probability prediction, the evaluation index of the lung cancer subtyping identification model is calculated as the second index. The first indicator is compared with the second indicator. If the first indicator is better than the second indicator, the cancer subtype corresponding to the foreground region and the corresponding similarity probability are selected as feedback data and fed back to the lung cancer subtype identification model. If the second indicator is better than the first indicator, the cancer subtype corresponding to the background region and the corresponding similarity probability are selected as feedback data and fed back to the lung cancer subtype identification model.

2. The lung cancer pathological tissue typing system based on deep semantic segmentation network according to claim 1, characterized in that, The cancer subtype corresponding to a pathological tissue slice image is determined based on the statistical value of the second probability, along with the predicted probability of that cancer subtype, including: Based on the cancer subtype, several categories are divided. Each category includes the cancer subtype type and the second probability of a corresponding pixel belonging to that cancer subtype. The second probability values ​​in each category are sorted, and the number of pixels with the highest second probability is counted, or the number of pixels with a second probability greater than a set second probability threshold is counted. The counted pixels are set as the subtype region. All pixels in the subtype region are classified according to their respective cancer subtypes to form subclasses. The average value of the second probability corresponding to the pixels in each subclass is calculated, and this average value is used as the predicted probability of the corresponding cancer subtype in each subclass.

3. The lung cancer pathological tissue typing system based on deep semantic segmentation network according to claim 1, characterized in that, The lung cancer subtype identification model includes: dividing randomly selected tissue slice images as samples into a training set, a validation set, and a test set; The lung cancer classification and identification model is trained by classifying samples using the training set. The trained lung cancer classification and identification model is then iteratively evaluated using the validation set to select the optimal lung cancer subtype identification model. Finally, the optimal lung cancer subtype identification model is evaluated using evaluation metrics on the test set. The training set includes: segmenting tissue slice images in the training set into a grid with pixels as the unit based on a deep semantic segmentation network model; and training the classification of samples based on the annotations by labeling the cancer subtypes corresponding to each pixel. The validation set includes: iteratively training the model using the errors of sample data in the validation set to obtain a model that fits the dataset reasonably, thus forming the optimal cancer subtyping identification model.

4. The lung cancer pathological tissue typing system based on deep semantic segmentation network according to claim 3, characterized in that, By labeling the cancer subtype corresponding to each pixel, including: In the first self-service annotation unit, relevant professionals perform manual annotation, and machine learning is used based on the manual annotation to form a self-service annotation model. The cancer subtype corresponding to each pixel is annotated according to the self-service annotation model. The second self-service annotation unit uses a self-service annotation model for initial annotation. Relevant professionals then review and correct any inaccurate areas annotated by the self-service annotation model.

5. The lung cancer pathological tissue typing system based on deep semantic segmentation network according to claim 1, characterized in that, The cancer classification includes: adenocarcinoma, squamous cell carcinoma, and small cell carcinoma; The sample types are surgical resection samples, biopsy samples, and cytological samples.

6. The lung cancer pathological tissue typing system based on deep semantic segmentation network according to claim 1, characterized in that, The deep semantic segmentation network technology uses DeepLab v3, and the feature extraction network is ResNet-50, which is used to output pixel-level recognition results.

7. The lung cancer pathological tissue typing system based on deep semantic segmentation network according to claim 3, characterized in that, The evaluation metrics include: AUC value, sensitivity, and specificity; The AUC value is the area under the ROC curve.

Images