Method, system and equipment for multi-subtype classification of lung diseases based on chest imaging

Abstract

The present invention discloses a method, system and device for multi-subtype classification of lung diseases based on chest images, which belongs to the classification of lung diseases in the field of artificial intelligence medical technology. The purpose is to solve the technical problem in the prior art that the loss function cannot make the model pay attention to the main pathological types and all pathological types in the data at the same time, resulting in low accuracy of multi-subtype classification by the model. It constructs a multi-subtype classification model and innovatively integrates three different losses during model training, which are used to enable the model to predict all existing pathological types, to enable the model to mainly learn to predict the main pathological types, and to enable the model to constrain the prototype vectors of each category to have discrimination, so that the model can better capture the association between different subtypes while learning the image representation of the main pathological subtypes, thereby improving the generalization ability of the model, enhancing the robustness of model training, alleviating the overfitting problem of the model, and improving the accuracy of the model for multi-subtype classification.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence medical technology and relates to the classification of lung diseases, and in particular to a method, system and device for classifying multiple subtypes of lung diseases based on chest images. Background Art

[0002] Lung adenocarcinoma is the most common histological subtype of non-small cell lung cancer, and its accurate classification and diagnosis are crucial for developing personalized treatment plans. Computed tomography (CT) plays a key role in the early detection of lung cancer, but the accuracy of existing imaging techniques in distinguishing different subtypes of lung adenocarcinoma still needs to be improved. This limitation may lead to deviations in treatment decisions, thereby affecting the patient's clinical prognosis.

[0003] According to the 2015 World Health Organization (WHO) lung cancer classification criteria, lung adenocarcinoma includes multiple histological subtypes, such as lepidic, acinar, papillary, micropapillary, and solid. These subtypes differ significantly in molecular characteristics, clinical manifestations, treatment response, and prognosis. However, existing conventional imaging methods are difficult to effectively distinguish these subtypes, increasing the complexity of clinical decision-making.

[0004] Studies have shown that different subtypes of lung adenocarcinoma have specific features in CT imaging, and these features are well correlated with pathological findings and prognostic indicators. Therefore, accurate analysis of lung adenocarcinoma subtypes based on CT images is expected to provide an important basis for patient classification diagnosis, prognosis assessment and individualized treatment under non-invasive conditions.

[0005] The invention patent application with application number 202111583548.X discloses a lung cancer subtype diagnosis system based on the fusion of image and gene features, which includes: a data acquisition module, which acquires the image data to be diagnosed and the gene data to be diagnosed of the same patient; an image feature extraction module, which extracts the image features to be diagnosed at different resolutions from the image data to be diagnosed; a gene feature extraction module, which extracts the gene features to be diagnosed from the gene data to be diagnosed; a feature fusion and subtype diagnosis module, which fuses the gene features to be diagnosed with the image features to be diagnosed at each resolution to obtain the fused features to be diagnosed at each resolution; diagnoses the fused features to be diagnosed at each resolution to obtain the diagnosis results at each resolution; and fuses the diagnosis results at all resolutions to obtain the lung cancer subtype diagnosis results of the current patient. First, the patient's image data is cut at different resolutions, and features are extracted and combined to obtain the image features of the patient at the corresponding resolution. Gene features are extracted through an attention-based gene feature extraction network, and the extracted gene features are fused with the image-level features combined at the corresponding resolution. Then, attention parameter layers based on the fused features of the corresponding resolution are constructed to capture the most correlated features between gene features and image features in the fused features of the corresponding resolution. The fused features are diagnosed through a diagnostic network based on the fused features of the corresponding resolution. Finally, the diagnostic results obtained by the fused features at each resolution are integrated to obtain the patient's final cancer subtype diagnosis result.

[0006] In addition, the invention patent with application number 202210271664.6 discloses a deep learning-based classification method for lung cancer CT image subtypes, which includes the following steps: S1, preprocessing: collecting the original data information of the CT image and preprocessing it to obtain an enhanced data set, and dividing the preprocessed CT image data into a test set and a training set; S2, building a model: building an improved 3D-LungNet network model for feature extraction of the preprocessed image; S3, training model: inputting the training set into the improved 3D-LungNet network model for training to obtain a trained CT image classification model; S4, test set input model: inputting the test set into the CT image classification model to obtain the CT image classification result. This method improves the feature extraction efficiency and image classification accuracy of the network layer by making full use of the intrinsic connection between medical images and natural images for feature migration, and reduces the redundant calculation amount in the training process. It also improves the classification accuracy and classification precision of CT images, effectively improving the diagnostic effect of CT images.

[0007] The above invention patent applications all involve building a network model for feature extraction and training the network model, and cross entropy is used as the loss function to optimize the model during training. For example, the invention patent application 202210271664.6 discloses a model training and optimization method, and its loss function is: .

[0008] However, when classifying subtypes of lung diseases, simply using an overall loss function to optimize the model can only focus on the main pathological types in the data and easily ignore all the pathological types in the data, or the model can only focus on all the pathological types in the data and easily ignore the main pathological types in the data, which ultimately results in low prediction accuracy of the network model for multiple subtypes of lung diseases and low multi-subtype classification accuracy. Summary of the invention

[0009] The purpose of the present invention is to solve the technical problem in the prior art that the loss function cannot make the model pay attention to the main pathological types and all pathological types in the data at the same time, resulting in low accuracy of multi-subtype classification of the model, and to provide a multi-subtype classification method, system and equipment for lung disease based on chest imaging.

[0010] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0011] A multi-subtype classification method for lung diseases based on chest images comprises the following steps:

[0012] Step 1: Obtain sample and label data;

[0013] Obtain CT sample images, obtain the pathology report corresponding to the lesions in the CT sample images, and extract the subtype information and proportion of each tissue in the pathology report to obtain the pathology label; mark the lesion location in the CT sample image to obtain label data;

[0014] Step 2, construct a multi-subtype classification model;

[0015] Construct a multi-subtype classification model for feature extraction and classification of CT images;

[0016] Step 3, training a multi-subtype classification model;

[0017] The multi-subtype classification model constructed in step 2 is trained using the CT sample images, label data, and subtype information and proportions of each tissue in the pathology report obtained in step 1;

[0018] During training, the loss function is:

[0019]

[0020]

[0021]

[0022]

[0023] in, represents the loss based on the proportion of all pathological components, represents the loss based on the main pathology type, represents the discrimination loss of the prototype vectors constraining each category, , , represents the weight of each loss, Indicates Category, represents the trainable prototype vector set for each category, Indicates The prototype vector of the category, Indicates The predicted probability of the category, represents the actual proportion of the nth pathological component, Indicates the predicted probability of the most common subtype category;

[0024] Step 4, real-time classification;

[0025] Obtain the CT image to be classified and input it into the multi-subtype classification model trained in step 3. The multi-subtype classification model outputs the multi-subtype classification result.

[0026] Furthermore, in step 1, when extracting the subtype information and proportion of each tissue in the pathology report, natural language technology is used to extract the information and proportion of each histological subtype in the pathology report to obtain a pathology label, which is expressed as:

[0027]

[0028]

[0029] in, represents the pathology label extracted from the pathology report, , , …, They represent the components corresponding to the eight subtypes, namely atypical adenomatous hyperplasia, adenocarcinoma in situ, minimally invasive adenocarcinoma, lepidic type, acinar type, papillary type, micropapillary type, and solid type. Each component represents the proportion of the subtype in the sample.

[0030] Furthermore, if one or more subtypes have no proportion, the proportion of the subtype / subtypes is reset to the same value, and the sum of the proportions of the 8 subtypes after the reset is equal to 1.

[0031] Furthermore, the classification labels of the eight subtypes are mapped one by one to the classification labels of the five subtypes, the classification labels of the three subtypes, and the classification labels of the two subtypes. The specific mapping relationship is:

[0032] .

[0033] Furthermore, the multi-subtype classification model includes an image feature extraction network and a classifier. The CT image is passed through the image feature extraction network to generate a feature vector , the classifier calculates the feature vector With Prototype vector of the category The cosine similarity between them is calculated, and the category with the highest cosine similarity is taken as the final predicted category.

[0034] Furthermore, cosine similarity The calculation formula is:

[0035]

[0036] in, represents the feature vector, Indicates The prototype vector of the category.

[0037] Further, in step 3, Prediction probability of category The calculation formula is:

[0038]

[0039] in, represents the feature vector, represents a one-dimensional learnable parameter, and the predicted probability belongs to the 8-dimensional real number space.

[0040] A multi-subtype classification system for lung disease based on chest imaging, including:

[0041] The sample and label data acquisition module is used to obtain CT sample images, obtain the pathology report corresponding to the lesion in the CT sample image, and extract the subtype information and proportion of each tissue in the pathology report to obtain the pathology label; mark the lesion location in the CT sample image to obtain label data;

[0042] A multi-subtype classification model building module is used to build a multi-subtype classification model for feature extraction and classification of CT images;

[0043] The multi-subtype classification model training module is used to train the multi-subtype classification model constructed by the multi-subtype classification model construction module using the CT sample images, label data, and subtype information and proportions of each tissue in the pathology report obtained by the sample and label data acquisition module;

[0044] During training, the loss function is:

[0045]

[0046]

[0047]

[0048]

[0049] in, represents the loss based on the proportion of all pathological components, represents the loss based on the main pathology type, represents the discrimination loss of the prototype vectors constraining each category, , , represents the weight of each loss, Indicates Category, represents the trainable prototype vector set for each category, Indicates The prototype vector of the category, Indicates The predicted probability of the category, represents the actual proportion of the nth pathological component, Indicates the predicted probability of the most common subtype category;

[0050] The real-time classification module is used to obtain the CT image to be classified and input the multi-subtype classification model trained by the multi-subtype classification model training module, and the multi-subtype classification model outputs the multi-subtype classification result.

[0051] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0052] The beneficial effects of the present invention are as follows:

[0053] In the present invention, three different losses are integrated during model training: To enable the model to predict all existing pathological types, It is used to let the model learn to predict the main pathological types. It is used to constrain the prototype vectors of each category to be discriminative by ,loss and loss By performing weighted averaging, the model can better capture the associations between different subtypes while learning the image representations of the main pathological subtypes, thereby improving the generalization ability of the model, enhancing the robustness of model training, alleviating the overfitting problem of the model, and improving the accuracy of the model in multi-subtype classification. This effectively solves the technical problem of low accuracy in multi-subtype classification caused by the loss function that makes it impossible for the model to simultaneously focus on the main pathological types and all pathological types in the data.

[0054] In addition, in the present invention, the pathological results of lung adenocarcinoma are used as the gold standard. Based on the characteristics of the pathological report of lung adenocarcinoma and the images of different subtype components on prognosis, a soft label is designed for training the classification model, so that the model can predict a variety of different subtypes in the lesions, which is convenient for better guidance of intervention and prognosis in subsequent practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a schematic diagram of the process of the present invention;

[0056] Figure 2 It is a prediction schematic diagram of the multi-subtype classification model in the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0058] Therefore, based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0059] Example 1

[0060] This embodiment provides a multi-subtype classification method for lung diseases based on chest images, which is used to complete the 2, 3, 5, and 8 classification tasks of 4 levels, including non-invasive-invasive, non-invasive-microinvasive-invasive, non-invasive-microinvasive-well-differentiated-moderately-differentiated-poorly differentiated, atypical adenomatous hyperplasia-adenocarcinoma in situ-microinvasive adenocarcinoma-lepidic-alveolar-papillary-micropapillary-solid, in a unified framework. Figure 1 As shown, the classification method specifically includes the following steps:

[0061] Step 1: Obtain sample and label data;

[0062] Obtain CT sample images, obtain the pathology report corresponding to the lesions in the CT sample images, extract the subtype information and proportion of each tissue in the pathology report, and obtain the pathology label; mark the lesion location in the CT sample image to obtain label data.

[0063] Several CT sample images were obtained from West China Hospital of Sichuan University, and experienced professional physicians annotated the lesion locations in the obtained CT sample images to obtain annotation boxes. In addition, the pathology report corresponding to the lesion in the CT sample image was obtained from West China Hospital of Sichuan University, and the pathology report was used as the gold standard.

[0064] In this embodiment, the subtype categories include 8 categories, namely, atypical adenomatous hyperplasia, adenocarcinoma in situ, microinvasive adenocarcinoma, lepidic type, acinar type, papillary type, micropapillary type, and solid type. The classification of these 8 subtypes is mapped one by one to 5 subtype classifications, 3 subtype classifications, and 2 subtype classifications. The specific mapping relationship is:

[0065] .

[0066] In this embodiment, when labeling subtype categories according to the pathology report, the labels are at the most fine-grained level (8 categories), and then the hierarchical mapping relationship of the categories themselves is used. Finally, each sample image has 2, 3, 5, or 8 labels.

[0067] For the acquired pathology report, the existing natural language technology is used (the technicians in this field can directly select and apply the existing technology according to the needs) to extract the information of each histological subtype and its proportion in the pathology report to obtain the pathology label. The pathology label is expressed as:

[0068]

[0069] Among them, the vector represents the pathology label extracted from the pathology report, , , …, They represent the components corresponding to the eight subtypes of atypical adenomatous hyperplasia, adenocarcinoma in situ, microinvasive adenocarcinoma, lepidic, acinar, papillary, micropapillary, and solid. Each component represents the proportion of the subtype in the sample, and the sum of the components of all eight subtypes is 100%, that is, .

[0070] In the pathology report, if one or more subtypes do not have a proportion, in order to ensure the accuracy of data processing, the proportion of this / these subtypes will be reset to the same value (if multiple subtypes do not have a proportion, the proportion of each subtype will be set to the same value), and the sum of the proportions of the 8 subtypes after the reset will be equal to 1.

[0071] In addition, use , which is the subtype classification with the largest proportion, as the main pathological subtype of this lesion.

[0072] For all the above data sets, they are divided into training set, validation set and test set according to the ratio of 70%, 15% and 15%. The training set is used for model training, the validation set is used for model tuning, and the test set is used to test the accuracy and generalization of the model.

[0073] Step 2, construct a multi-subtype classification model;

[0074] A multi-subtype classification model was constructed for feature extraction and classification of CT images.

[0075] The multi-subtype classification model includes an image feature extraction network and a classifier. The image feature extraction network adopts a network model in the prior art, including but not limited to a CNN network (ResNet, DenseNet, ConvNeXt), a transformer architecture (ViT, SwinViT) or a hybrid architecture. The classifier can also adopt a classifier in the prior art. The multi-subtype classification model is applicable to images of multiple dimensions, such as 2D, 3D or mixed dimensions; it is also applicable to various input image preprocessing methods, including but not limited to different cropping, scaling, normalization methods, and whether to use nodule masks, multi-channel or multi-scale images as additional channel inputs.

[0076] When the multi-subtype classification model is used for prediction, the specific prediction process is as follows: Figure 2 As shown; the CT image first passes through the image feature extraction network to generate a feature vector , and then the classifier calculates the feature vector With Prototype vector of the category The cosine similarity between them is calculated, and the category with the highest cosine similarity is taken as the final predicted category. If multiple categories have high similarity at the same time, it means that the lesion may have the characteristics of multiple different pathological subtypes.

[0077] In calculating the eigenvector With Prototype vector of the category When the cosine similarity between The calculation formula is:

[0078]

[0079] in, represents the feature vector, Indicates The prototype vector of the category.

[0080] Step 3, training a multi-subtype classification model;

[0081] The multi-subtype classification model constructed in step 2 is trained using the CT sample images, label data, and subtype information and proportions of each tissue in the pathology report obtained in step 1;

[0082] During training, the loss function is:

[0083]

[0084]

[0085]

[0086]

[0087]

[0088] in, represents the loss based on the proportion of all pathological components, represents the loss based on the main pathology type, represents the discrimination loss of the prototype vectors constraining each category, , , represents the weight of each loss, Indicates Category, represents the trainable prototype vector set for each category, Indicates The prototype vector of the category, Indicates The predicted probability of the category, represents the actual proportion of the nth pathological component, represents the predicted probability of the most common subtype category, represents the feature vector, represents a one-dimensional learnable parameter, and the predicted probability belongs to the 8-dimensional real number space.

[0089] It is the cross entropy calculated based on the soft labels (the proportion of all pathological components); It is the cross entropy calculated based on hard labels (main pathological types); the acquisition of the two labels can be seen in the data acquisition part; Let the algorithm mainly learn to predict the main pathological types, Allowing the algorithm to predict all existing pathology types. It is a constraint on the model itself (which can be understood as a model regularization). The purpose is to improve the model's fitting ability and reduce the risk of overfitting by constraining the distribution of feature vectors in space. This loss intends to minimize the cosine similarity between prototype vectors of different categories, so that the features of each category are evenly distributed in space, improving the model's fitting ability. The three losses are synergistic, through weighted averaging, acting on training and optimization from multiple aspects, helping the model to better capture the association between different subtypes while learning the image representation of the main pathological subtypes, and improving the generalization performance of the model.

[0090] During training, this embodiment adopts a bottom-up hierarchical label aggregation method to combine the model's 8-category prediction value with the label L and Mapped to 5-class, 3-class, and 2-class tasks, the corresponding prediction values ​​and gold standards are formed. After obtaining the prediction probability of the 8-class (finest granularity) level, the prediction of a coarser granularity level can be obtained through probability aggregation, as follows:

[0091] For the 5-class level, add the 8-class prediction probabilities and label vectors belonging to the same 5-class category;

[0092] For the 3-class level, add the 5-class prediction probabilities (or directly use the 8-class prediction probabilities) and label vectors (or directly use the 8-class label vectors) belonging to the same 3-class category;

[0093] For the 2-class level, add the 3-class prediction probabilities (or directly use the 8-class prediction probabilities) and label vectors (or directly use the 8-class label vectors) belonging to the same 2-class category.

[0094] This method ensures the consistency of predictions, that is, the prediction probability of a coarse-grained category is equal to the sum of the prediction probabilities of all the fine-grained categories it contains, and the prediction of a coarse-grained category can be naturally obtained through the prediction of a fine-grained category.

[0095] Step 4, real-time classification;

[0096] Obtain the CT image to be classified and input it into the multi-subtype classification model trained in step 3. The multi-subtype classification model outputs the multi-subtype classification result.

[0097] Example 2

[0098] This embodiment provides a multi-subtype classification system for lung diseases based on chest images, which specifically includes:

[0099] The sample and label data acquisition module is used to obtain CT sample images, obtain the pathology report corresponding to the lesion in the CT sample image, and extract the subtype information and proportion of each tissue in the pathology report to obtain the pathology label; mark the lesion location in the CT sample image to obtain label data.

[0100] Several CT sample images were obtained from West China Hospital of Sichuan University, and experienced professional physicians annotated the lesion locations in the obtained CT sample images to obtain annotation boxes. In addition, the pathology report corresponding to the lesion in the CT sample image was obtained from West China Hospital of Sichuan University, and the pathology report was used as the gold standard.

[0101] In this embodiment, the subtype categories include 8 categories, namely, atypical adenomatous hyperplasia, adenocarcinoma in situ, microinvasive adenocarcinoma, lepidic type, acinar type, papillary type, micropapillary type, and solid type. The classification of these 8 subtypes is mapped one by one to 5 subtype classifications, 3 subtype classifications, and 2 subtype classifications. The specific mapping relationship is:

[0102] .

[0103] In this embodiment, when labeling subtype categories according to the pathology report, the labels are at the most fine-grained level (8 categories), and then the hierarchical mapping relationship of the categories themselves is used. Finally, each sample image has 2, 3, 5, or 8 labels.

[0104] For the acquired pathology report, the existing natural language technology is used (the technicians in this field can directly select and apply the existing technology according to the needs) to extract the information of each histological subtype and its proportion in the pathology report to obtain the pathology label. The pathology label is expressed as:

[0105]

[0106] Among them, the vector represents the pathology label extracted from the pathology report, , , …, They represent the components corresponding to the eight subtypes of atypical adenomatous hyperplasia, adenocarcinoma in situ, microinvasive adenocarcinoma, lepidic, acinar, papillary, micropapillary, and solid. Each component represents the proportion of the subtype in the sample, and the sum of the components of all eight subtypes is 100%, that is, .

[0107] In the pathology report, if one or more subtypes do not have a proportion, in order to ensure the accuracy of data processing, the proportion of this / these subtypes will be reset to the same value (if multiple subtypes do not have a proportion, the proportion of each subtype will be set to the same value), and the sum of the proportions of the 8 subtypes after the reset will be equal to 1.

[0108] In addition, use , which is the subtype classification with the largest proportion, as the main pathological subtype of this lesion.

[0109] For all the above data sets, they are divided into training set, validation set and test set according to the ratio of 70%, 15% and 15%. The training set is used for model training, the validation set is used for model tuning, and the test set is used to test the accuracy and generalization of the model.

[0110] The multi-subtype classification model building module is used to build a multi-subtype classification model for feature extraction and classification of CT images.

[0111] The multi-subtype classification model includes an image feature extraction network and a classifier. The image feature extraction network adopts a network model in the prior art, including but not limited to a CNN network (ResNet, DenseNet, ConvNeXt), a transformer architecture (ViT, SwinViT) or a hybrid architecture. The classifier can also adopt a classifier in the prior art. The multi-subtype classification model is applicable to images of multiple dimensions, such as 2D, 3D or mixed dimensions; it is also applicable to various input image preprocessing methods, including but not limited to different cropping, scaling, normalization methods, and whether to use nodule masks, multi-channel or multi-scale images as additional channel inputs.

[0112] When the multi-subtype classification model is used for prediction, the specific prediction process is as follows: Figure 2 As shown; the CT image first passes through the image feature extraction network to generate a feature vector , and then the classifier calculates the feature vector With Prototype vector of the category The cosine similarity between them is calculated, and the category with the highest cosine similarity is taken as the final predicted category. If multiple categories have high similarity at the same time, it means that the lesion may have the characteristics of multiple different pathological subtypes.

[0113] In calculating the eigenvector With Prototype vector of the category When the cosine similarity between The calculation formula is:

[0114]

[0115] in, represents the feature vector, Indicates The prototype vector of the category.

[0116] The multi-subtype classification model training module is used to train the multi-subtype classification model constructed by the multi-subtype classification model construction module using the CT sample images, label data, and subtype information and proportions of each tissue in the pathology report obtained by the sample and label data acquisition module;

[0117] During training, the loss function is:

[0118]

[0119]

[0120]

[0121]

[0122]

[0123] in, represents the loss based on the proportion of all pathological components, represents the loss based on the main pathology type, represents the discrimination loss of the prototype vectors constraining each category, , , represents the weight of each loss, Indicates Category, represents the trainable prototype vector set for each category, Indicates The prototype vector of the category, Indicates The predicted probability of the category, represents the actual proportion of the nth pathological component, represents the predicted probability of the most common subtype category, represents the feature vector, represents a one-dimensional learnable parameter, and the predicted probability belongs to the 8-dimensional real number space.

[0124] It is the cross entropy calculated based on the soft labels (the proportion of all pathological components); It is the cross entropy calculated based on hard labels (main pathological types); the acquisition of the two labels can be seen in the data acquisition part; Let the algorithm mainly learn to predict the main pathological types, Allowing the algorithm to predict all existing pathology types. It is a constraint on the model itself (which can be understood as a model regularization). The purpose is to improve the model's fitting ability and reduce the risk of overfitting by constraining the distribution of feature vectors in space. This loss intends to minimize the cosine similarity between prototype vectors of different categories, so that the features of each category are evenly distributed in space, improving the model's fitting ability. The three losses are synergistic, through weighted averaging, acting on training and optimization from multiple aspects, helping the model to better capture the association between different subtypes while learning the image representation of the main pathological subtypes, and improving the generalization performance of the model.

[0125] During training, this embodiment adopts a bottom-up hierarchical label aggregation method to combine the model's 8-category prediction value with the label L and Mapped to 5-class, 3-class, and 2-class tasks, the corresponding prediction values ​​and gold standards are formed. After obtaining the prediction probability of the 8-class (finest granularity) level, the prediction of a coarser granularity level can be obtained through probability aggregation, as follows:

[0126] For the 5-class level, add the 8-class prediction probabilities and label vectors belonging to the same 5-class category;

[0127] For the 3-class level, add the 5-class prediction probabilities (or directly use the 8-class prediction probabilities) and label vectors (or directly use the 8-class label vectors) belonging to the same 3-class category;

[0128] For the 2-class level, add the 3-class prediction probabilities (or directly use the 8-class prediction probabilities) and label vectors (or directly use the 8-class label vectors) belonging to the same 2-class category.

[0129] This method ensures the consistency of predictions, that is, the prediction probability of a coarse-grained category is equal to the sum of the prediction probabilities of all the fine-grained categories it contains, and the prediction of a coarse-grained category can be naturally obtained through the prediction of a fine-grained category.

[0130] The real-time classification module is used to obtain the CT image to be classified and input the multi-subtype classification model trained by the multi-subtype classification model training module, and the multi-subtype classification model outputs the multi-subtype classification result.

[0131] Example 3

[0132] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of a multi-subtype classification method for lung diseases based on chest images.

[0133] The computer device may be a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with the user through a keyboard, a mouse, a remote control, a touch pad, or a voice control device.

[0134] The memory includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or D interface display memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. In other embodiments, the memory can also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory is often used to store the operating system and various application software installed on the computer device, such as the program code of the multi-subtype classification method of lung diseases based on chest imaging, etc. In addition, the memory can also be used to temporarily store various types of data that have been output or are to be output.

[0135] The processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor is generally used to control the overall operation of the computer device. In this embodiment, the processor is used to run the program code stored in the memory or process data, such as running the program code of the multi-subtype classification method for lung diseases based on chest images.

[0136] Example 4

[0137] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of a method for multi-subtype classification of lung diseases based on chest images.

[0138] The computer-readable storage medium stores an interface display program, and the interface display program can be executed by at least one processor so that the at least one processor executes the steps of the above-mentioned chest image-based multi-subtype classification method for lung diseases.

[0139] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server or network device, etc.) to execute the multi-subtype classification method of lung diseases based on chest imaging described in the embodiment of the present application.

Claims

1. A multi-subtype classification method for lung diseases based on chest images, characterized in that: The following steps are involved: Step 1: Obtain sample and label data; Obtain CT sample images, obtain the pathology report corresponding to the lesions in the CT sample images, and extract the subtype information and proportion of each tissue in the pathology report to obtain the pathology label; Mark the lesion location in the CT sample image to obtain label data; Step 2, construct a multi-subtype classification model; Construct a multi-subtype classification model for feature extraction and classification of CT images; Step 3, training a multi-subtype classification model; The multi-subtype classification model constructed in step 2 is trained using the CT sample images, label data, and subtype information and proportions of each tissue in the pathology report obtained in step 1; During training, the loss function is: in, represents the loss based on the proportion of all pathological components, represents the loss based on the main pathology type, represents the discrimination loss of the prototype vectors constraining each category, , , represents the weight of each loss, Indicates Category, represents the trainable prototype vector set for each category, Indicates The prototype vector of the category, Indicates The predicted probability of the category, represents the actual proportion of the nth pathological component, Indicates the predicted probability of the most common subtype category; Step 4, real-time classification; Obtain a CT image to be classified and input it into the multi-subtype classification model trained in step 3, and the multi-subtype classification model outputs a multi-subtype classification result; In step 2, the multi-subtype classification model includes an image feature extraction network and a classifier. The CT image is passed through the image feature extraction network to generate a feature vector , the classifier calculates the feature vector With Prototype vector of the category The cosine similarity between them is calculated, and the category with the highest cosine similarity is taken as the final predicted category.

2. The method for multi-subtype classification of lung diseases based on chest imaging according to claim 1, characterized in that: In step 1, when extracting the subtype information and proportion of each tissue in the pathology report, natural language technology is used to extract the information and proportion of each histological subtype in the pathology report to obtain the pathology label, which is expressed as: in, represents the pathology label extracted from the pathology report, , , …, They represent the components corresponding to the eight subtypes, namely atypical adenomatous hyperplasia, adenocarcinoma in situ, minimally invasive adenocarcinoma, lepidic type, acinar type, papillary type, micropapillary type, and solid type. Each component represents the proportion of the subtype in the sample.

3. The method for multi-subtype classification of lung diseases based on chest imaging as claimed in claim 2, characterized in that: If one or more subtypes have no proportion, the proportion of the subtype / subtypes will be reset to the same value, and the sum of the proportions of the 8 subtypes after the reset is equal to 1.

4. The method for multi-subtype classification of lung diseases based on chest imaging according to claim 2, characterized in that: The classification labels of the eight subtypes are mapped one by one to the classification labels of the five subtypes, the classification labels of the three subtypes, and the classification labels of the two subtypes. The specific mapping relationship is as follows: 。 5. The method for multi-subtype classification of lung diseases based on chest imaging according to claim 1, characterized in that: Cosine similarity The calculation formula is: in, represents the feature vector, Indicates The prototype vector of the category.

6. The method for multi-subtype classification of lung diseases based on chest imaging according to claim 1, characterized in that: In step 3, Prediction probability of category The calculation formula is: in, represents the feature vector, represents a one-dimensional learnable parameter, Indicates that the predicted probability belongs to the 8-dimensional real number space.

7. A multi-subtype classification system for lung diseases based on chest imaging, characterized in that: include: The sample and label data acquisition module is used to obtain CT sample images, obtain the pathology report corresponding to the lesion in the CT sample image, and extract the subtype information and proportion of each tissue in the pathology report to obtain the pathology label; Mark the lesion location in the CT sample image to obtain label data; A multi-subtype classification model building module is used to build a multi-subtype classification model for feature extraction and classification of CT images; The multi-subtype classification model training module is used to train the multi-subtype classification model constructed by the multi-subtype classification model construction module using the CT sample images, label data, and subtype information and proportions of each tissue in the pathology report obtained by the sample and label data acquisition module; During training, the loss function is: in, represents the loss based on the proportion of all pathological components, represents the loss based on the main pathology type, represents the discrimination loss of the prototype vectors constraining each category, , , represents the weight of each loss, Indicates Category, represents the trainable prototype vector set for each category, Indicates The prototype vector of the category, Indicates The predicted probability of the category, represents the actual proportion of the nth pathological component, Indicates the predicted probability of the most common subtype category; A real-time classification module is used to obtain a CT image to be classified and input the multi-subtype classification model trained by the multi-subtype classification model training module, and the multi-subtype classification model outputs a multi-subtype classification result; In the multi-subtype classification model construction module, the multi-subtype classification model includes an image feature extraction network and a classifier. The CT image generates a feature vector after passing through the image feature extraction network. , the classifier calculates the feature vector With Prototype vector of the category The cosine similarity between them is calculated, and the category with the highest cosine similarity is taken as the final predicted category.

8. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 6.

Images