Interstitial lung disease classification method and system based on multi-dimensional feature tags

Abstract

The invention discloses an interstitial lung disease classification method based on a multi-dimensional feature tag, and the method comprises the steps: obtaining lung CT image information which comprises a CT image and associated clinical information; performing region segmentation on the CT image to obtain three-dimensional space information of each region; based on the three-dimensional space information of each region, performing target feature detection on the CT image to obtain a three-dimensional morphological feature of a target feature; performing depth feature extraction and mathematical calculation on the target features to obtain high-order mathematical features; fusing the three-dimensional morphological features and the high-order mathematical features to generate high-dimensional fused image features; encoding the associated clinical information, and generating a multi-dimensional feature tag in combination with the high-dimensional fusion image features; and generating a classification result of the interstitial lung disease through a preset classification model based on the multi-dimensional feature tag. Through multi-dimensional feature analysis from an image level and a clinical level, rapid and accurate classification of various interstitial lung diseases can be realized.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, to an interstitial lung disease classification method and system based on multi-dimensional feature tags. Background Art

[0002] Interstitial lung disease (ILD) is a group of diseases mainly characterized by lesions in the alveolar wall and including the surrounding tissues and their adjacent supporting structures. The main interstitial lung diseases are divided into several types. For example, they can be roughly divided into UIP, NSIP, OP, pUIP, etc. How to quickly determine the type of interstitial lung disease helps to determine the treatment plan in time and improve the diagnostic efficiency.

[0003] Existing ILD classification methods mainly rely on doctors' empirical judgments of images or medical image retrieval with the help of visual models. However, due to the diverse manifestations of interstitial lung diseases in imaging examinations, including honeycomb-like, reticular, ground-glass-like, etc., and the lack of specificity and easy overlap in the imaging manifestations of various interstitial lung diseases, it is difficult for existing classification methods to quickly determine the category of interstitial lung diseases. Summary of the Invention

[0004] The purpose of the present application is to provide an interstitial lung disease classification method and system based on multi-dimensional feature tags. In addition to deeply analyzing the lesion characteristics of interstitial lung diseases in different dimensions, the associated clinical information is also effectively fused to form multi-dimensional feature tags. Taking the multi-dimensional feature tags as the target, the interstitial lung disease can be accurately classified quickly.

[0005] In the first aspect, the present application provides an interstitial lung disease classification method based on multi-dimensional feature tags, adopting the following technical solutions: Obtain pulmonary CT image information, where the CT image information includes CT images and associated clinical information; Perform regional segmentation on the CT images through a preset three-dimensional model to obtain the three-dimensional spatial information of each region; Based on the three-dimensional spatial information of each region, perform target feature detection on the CT images to obtain the three-dimensional morphological features of the target features; Perform deep feature extraction and mathematical calculations on the target features to obtain high-order mathematical features; Fuse the three-dimensional morphological features and high-order mathematical features to generate high-dimensional fused image features; Encode the associated clinical information and combine it with the high-dimensional fused image features to generate multi-dimensional feature tags; Based on the multi-dimensional feature tags, generate the classification result of the interstitial lung disease through a preset classification model.

[0006] Through the above technical solutions, the three-dimensional morphological features, depth features, and mathematical features of the lesion characteristics can be fused, and the lesion characteristics can be analyzed and modeled from multiple dimensions, enhancing the description of the correlation information of the lesion characteristics from the whole to the local, contributing to the rapid and accurate classification of various interstitial lung diseases. At the same time, integrating clinical information can further improve the classification efficiency.

[0007] Optionally, the three-dimensional morphological features include morphological feature information and spatial occupancy information. Based on the three-dimensional spatial information of each region, performing target feature detection on the CT image to obtain the three-dimensional morphological features of the target feature includes: Performing target feature detection on the CT image to obtain the morphological feature information and three-dimensional spatial information of the target feature; Based on the three-dimensional spatial information of each region, matching the spatial position information of the target feature in the associated region; Based on the three-dimensional spatial information of each region and the target feature, respectively calculating and obtaining the volume of each region and the volume of the target feature; Based on the volume of each region and the volume of the target feature, calculating and obtaining the spatial occupancy information of the target feature through the spatial position information of the target feature in the associated region.

[0008] Optionally, extracting the depth features of the target feature to obtain high-order mathematical features includes: Based on the target feature, extracting depth features through a preset deep neural network; Performing mathematical calculations on the depth features to obtain mathematical features; Fusing the depth features and the mathematical features to generate high-order mathematical features.

[0009] Optionally, fusing the three-dimensional morphological features and the high-order mathematical features to generate high-dimensional fused image features includes: Based on the three-dimensional morphological features and high-order mathematical features, obtaining associated features through a preset association model; Generating a fusion weight based on the associated features; Based on the fusion weight, fusing the three-dimensional morphological features and the high-order mathematical features to generate high-dimensional fused image features.

[0010] Optionally, the associated clinical information includes off-table information and in-table information. Encoding the associated clinical information and combining it with the high-dimensional fused image features to generate multi-dimensional feature labels includes: Through a preset data analysis model, obtaining the influence coefficient of the off-table information on various interstitial lung diseases and generating a first influence vector; Through a preset disease diagnosis library, obtaining the matching degree of the in-table information with various interstitial lung diseases and generating a second influence vector; Encode the clinical information based on the first influence vector and the second influence vector to generate clinical feature labels; Integrate the clinical feature labels with the high-dimensional fused image features to generate multi-dimensional feature labels.

[0011] Optionally, after generating the classification result of interstitial lung disease through a preset classification model based on the multi-dimensional feature labels, it further includes: Obtain dynamic evolution features through a preset dynamic evolution model according to the classification result; Perform three-dimensional reconstruction on the current CT image based on the dynamic evolution features to generate a new CT image; Add the new CT image to the preset training database.

[0012] Optionally, the obtaining dynamic evolution features through a preset dynamic evolution model according to the classification result includes: Match the corresponding preset dynamic evolution model from the preset dynamic evolution database according to the classification result; Generate variable features through a preset dynamic evolution model based on the current CT image and the target features; Modify the variable features based on the associated clinical information to generate dynamic evolution features.

[0013] Optionally, after adding the new CT image to the preset training database, it further includes: Determine whether the data volume of the new CT image added to the current preset training database reaches a preset update threshold; If so, perform iterative update on the preset classification model based on the current preset training database to generate a new preset classification model.

[0014] In a second aspect, the present application provides an interstitial lung disease classification system based on multi-dimensional feature labels, including: A data acquisition module 101 for acquiring pulmonary CT image information, where the CT image information includes CT images and associated clinical information; A region segmentation module 102 for performing region segmentation on the CT image through a preset three-dimensional model to obtain the three-dimensional spatial information of each region; A feature detection module 103 for performing target feature detection on the CT image based on the three-dimensional spatial information of each region to obtain the three-dimensional morphological features of the target features; A multi-dimensional feature fusion module 104 for performing deep feature extraction and mathematical calculations on the target features to obtain high-order mathematical features, fusing the three-dimensional morphological features and the high-order mathematical features to generate high-dimensional fused image features, encoding the associated clinical information, and combining with the high-dimensional fused image features to generate multi-dimensional feature labels; The classification result generation module 105 is configured to generate a classification result of interstitial lung disease based on multi-dimensional feature tags through a preset classification model.

[0015] In a third aspect, the present application provides a computer-readable storage medium storing a computer program that can be loaded and executed by a processor to perform the above-mentioned method for classifying interstitial lung disease based on multi-dimensional feature tags.

[0016] In summary, the present application first fuses the three-dimensional morphological features, depth features, and mathematical features of lesion features at the imaging level, and combines clinical information to construct multi-dimensional feature tags, analyzes and models the lesion features from multiple dimensions, increases the differential expression of features, and helps to quickly and accurately classify various interstitial lung diseases. In addition, when fusing the morphological features and mathematical features of lesion features, by constructing an association model, the internal connection of lesion features in different manifestation forms can be mined, and the expression of lesion features is further enhanced, which can improve the classification efficiency. Furthermore, by constructing a dynamic evolution model, effective data augmentation can be performed on the data participating in classification based on the evolution law of lesion features, which is convenient for subsequent iterative optimization of the classification model to enhance the generalization ability of the classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flowchart of a method for classifying interstitial lung disease based on multi-dimensional feature tags provided by an embodiment of the present application; Figure 2 is an example diagram of a pulmonary CT image provided by an embodiment of the present application; Figure 3 is a flowchart of obtaining the three-dimensional morphological features of target features provided by an embodiment of the present application; Figure 4 is a flowchart of extracting depth features of target features and obtaining high-order mathematical features provided by an embodiment of the present application; Figure 5 is a flowchart of generating multi-dimensional feature tags provided by an embodiment of the present application; Figure 6 is an example diagram of the classification effect of interstitial lung disease provided by an embodiment of the present application; Figure 7 is a schematic diagram of a system for classifying interstitial lung disease based on multi-dimensional feature tags provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The following further describes the present application in detail with reference to the attached Figure 1 - attached Figure 7 , and makes a further detailed description of the present application.

[0019] The present application provides a method for classifying interstitial lung disease based on multi-dimensional feature tags. Refer toFigure 1 , including the following steps: S100. Obtain pulmonary CT image information.

[0020] Among them, the CT image information includes CT images and associated clinical information. The CT image is a three-dimensional DICOM image. DICCOM is a commonly used medical data imaging format. In addition to containing image pixel data, it also contains CT scan information and spatial positioning information of the image, which can help with three-dimensional reconstruction of the image to observe the pathological conditions of the lungs from different angles and planes; the associated clinical information includes data information such as height, weight, age, and lung function.

[0021] For the original pulmonary CT image, see Figure 2 . After obtaining the original CT image, corresponding preprocessing will also be performed on the original CT image. For example, the image size is adjusted by cropping and scaling the image according to the network model parameters, that is, the CT image here is the preprocessed image.

[0022] S200. Perform region segmentation on the CT image through a preset three-dimensional model to obtain the three-dimensional spatial information of each region.

[0023] Because interstitial lung disease is often accompanied by changes in the morphology of the lung lobes, abnormal density of the lung parenchyma, etc., by observing the characteristic changes of each lung lobe, it is helpful to detect the subtle pathological changes of the disease at an early stage. In addition, the disease states of different types of interstitial lung diseases also vary in the performance of each region.

[0024] Therefore, in the embodiment of the present application, after obtaining the pulmonary CT image, region segmentation will be performed on the CT image first to obtain the three-dimensional spatial information of the left lung, right lung, and regions of the left upper lobe, left lower lobe, right upper lobe, right middle lobe, and right lower lobe of the lung. The three-dimensional spatial information here characterizes the spatial distribution and morphological size of each region in the lung.

[0025] The preset three-dimensional model here is a lung region segmentation model generated through training with the lung parenchyma, lung fissures, and each lung lobe region as annotation information, using a three-dimensional depth model as the architecture through a large amount of pulmonary CT image data, such as commonly used model architectures like 3D-UNet and VNet.

[0026] S300. Based on the three-dimensional spatial information of each region, perform target feature detection on the CT image to obtain the three-dimensional morphological features of the target features.

[0027] Among them, the target feature refers to the lesion feature of interstitial lung disease. For example, visual features such as honeycombing, reticulation, ground-glass opacity, consolidation, and emphysema; the three-dimensional morphological feature characterizes the morphological feature and distribution feature of the lesion feature, that is, the three-dimensional morphological feature includes morphological feature information and spatial occupancy information. The morphological feature information refers to information such as the shape, size, concavity and convexity, contour, and edge of the lesion feature; the spatial occupancy information refers to the spatial position, volume size of the lesion feature, and the volume occupancy in each associated region.

[0028] Since the visual feature manifestations of different types of interstitial lung diseases vary, clarifying the distribution of lesion features in each region of the lungs helps to accurately determine the category of interstitial lung disease.

[0029] In the embodiment of the present application, based on the three-dimensional spatial information of each region, target feature detection is performed on the CT image to obtain the three-dimensional morphological feature of the target feature. See Figure 3 , which specifically includes the following steps: S310. Perform target feature detection on the CT image to obtain the morphological feature information and three-dimensional spatial information of the target feature.

[0030] S320. Based on the three-dimensional spatial information of each region, match the spatial position information of the target feature in the associated region.

[0031] S330. Based on the three-dimensional spatial information of each region and the target feature, calculate and obtain the volume of each region and the volume of the target feature respectively.

[0032] S340. Based on the volume of each region and the volume of the target feature, calculate and obtain the spatial occupancy information of the target feature through the spatial position information of the target feature in the associated region.

[0033] In the embodiment of the present application, first, target feature detection is performed on the CT image to obtain the morphological feature information and three-dimensional spatial information of the target feature.

[0034] The feature detection of CT images mentioned here is similar to the above-mentioned regional segmentation of CT images through a preset three-dimensional model to obtain the three-dimensional spatial information of each region. Both use a preset three-dimensional model, but the detection targets are different. Regional segmentation uses the detection of lung parenchyma and pulmonary fissures as the benchmark division markers, while obtaining the target features, that is, lesion features, uses visual features such as honeycombing, reticulation, ground-glass, consolidation, and emphysema as the detection targets for image detection, and marks the positions of the detected targets. That is, the three-dimensional spatial information of the target features can also be obtained. The morphological feature information is obtained by further morphological processing of the target features on the basis of detecting the target features. For example, instance segmentation algorithms such as MaskR-CNN are used to subdivide the target features and obtain morphological information such as the shape, size, concavity, convexity, contour, and edge of the target features through mask operations. Fine detection of the morphology of the lesions also helps to classify and determine the types of interstitial lung diseases.

[0035] In order to determine the distribution of lesion features in each region of the lungs, by combining the three-dimensional spatial information of each region and the three-dimensional spatial information of the target features, spatial position relationship matching can be performed to determine which region or regions the target features are located in, that is, the so-called associated regions. Thus, the spatial position relationship between the target features and the associated regions can be obtained, that is, the spatial position information of the target features in the associated regions.

[0036] In addition, considering the volume of each lung lobe region and the volume ratio of the lesion features can quantitatively evaluate the involvement range and severity of the lesions of interstitial lung diseases, which also helps to classify interstitial lung diseases.

[0037] Therefore, after obtaining the three-dimensional spatial information of each region and the three-dimensional spatial information of the target features, the volume sizes of each region and the volume size of the target features can be calculated respectively. And with the help of the spatial position information of the target region in the associated region, the spatial occupancy ratio of the target features can be calculated to help evaluate the lesion degree and subsequent lesion trend of the lesion features, and at the same time, it can also assist in determining the types of interstitial lung diseases.

[0038] For example, for lung CT images, through the above operations, the volume sizes of each region and the spatial occupancy ratio information of the target features obtained are shown in Table 1.

[0039] Table 1: Among them, "normal" represents the remaining normal tissues in the lung region except for the lesion features. The data in the table can intuitively reflect the main position distribution and spatial occupancy ratio information of each lesion feature in the entire lung region.

[0040] S400. Perform deep feature extraction and mathematical calculation on the target feature to obtain high-order mathematical features.

[0041] Regarding the classification of interstitial lung diseases, whether it is the morphological feature information or the spatial proportion information of the lesion features, they are all low-dimensional judgment references. In the face of the situation where the imaging manifestations of various interstitial lung diseases lack specificity and are prone to overlap, they are not sufficient as the basis for classification judgment.

[0042] Therefore, in the embodiments of this application, deeper feature mining and analysis will also be performed on the target feature, that is, the lesion feature. Specifically, deep feature extraction and mathematical calculation are performed on the target feature to obtain high-order mathematical features.

[0043] Specifically, to perform deep feature extraction on the target feature and obtain high-order mathematical features, refer to Figure 4 , which specifically includes the following steps: S410. Based on the target feature, extract deep features through a preset deep neural network.

[0044] S420. Perform mathematical calculation on the deep features to obtain mathematical features.

[0045] S430. Integrate the deep features and the mathematical features to generate high-order mathematical features.

[0046] Among them, deep features refer to the feature representations learned through deep neural networks. They can capture complex and abstract feature expressions and semantic information in images and have good description ability for the overall structure and semantic content of images. For example, in lung CT images, deep features can learn the comprehensive features corresponding to different types of interstitial lung diseases in terms of texture, shape, density, etc., which helps to distinguish different categories of interstitial lung diseases macroscopically.

[0047] High-order mathematical features are more statistical and structural features extracted through specific mathematical calculation methods on the basis of deep features. For example, the gray-level co-occurrence matrix can describe the spatial correlation of pixel gray values in an image and reflect information such as the thickness and directionality of the texture. These high-order mathematical features can more finely depict the local characteristics and microscopic structures of images, are more sensitive to the subtle features and changes of diseases, and help to distinguish different categories of interstitial lung diseases microscopically.

[0048] First, deep features will be extracted from the target feature through a preset deep neural network, that is, using the data representation of the target feature as the input and adopting a preset deep neural network for deep feature extraction to obtain deep features. Here, the preset deep neural network is equivalent to a conventional convolutional neural network for multi-layer convolution processing, such as using network structures like GGNet and ResNet.

[0049] Next, mathematical calculations are performed on the depth features to obtain mathematical features. For example, statistical calculations are performed on the extracted depth features, such as calculating the mean, variance, standard deviation, etc., which can reflect the distribution of the depth features; calculating higher-order statistical quantities, such as skewness, kurtosis, and covariance matrix, etc., can measure the degree of asymmetry of the feature distribution and the correlation between different depth features.

[0050] Finally, the depth features and mathematical features are fused to generate high-order mathematical features, which are used as a reference for subsequent judgment of the interstitial lung disease category.

[0051] The fusion here can adopt simple feature splicing, splicing the extracted depth features and the calculated mathematical features to form a new feature vector, or feature weighted fusion, assigning different weights to the depth features and mathematical features, and then performing weighted fusion. Of course, an attention mechanism can also be introduced to allow the model to automatically learn the importance of the depth features and mathematical features, and fuse the features according to the attention weights. This application does not make specific limitations.

[0052] S500. Fuse the three-dimensional morphological features and high-order mathematical features to generate high-dimensional fused image features.

[0053] Since there may be a certain correlation between the three-dimensional morphological features and high-order mathematical features. For example, morphological information such as the roughness of the edge in the three-dimensional morphology of the lesion may be related to the texture complexity reflected by certain statistics in the high-order mathematical features. By establishing an internal connection, in addition to discovering the complementary parts of the two features in describing the lesion, the representation of the related features can also be strengthened.

[0054] Therefore, in the embodiments of this application, when fusing the three-dimensional morphological features and high-order mathematical features, a correlation analysis is also performed on the three-dimensional morphological features and high-order mathematical features to further enhance the description of the lesion features.

[0055] Specifically, fusing the three-dimensional morphological features and high-order mathematical features to produce high-dimensional fused image features includes the following steps: S510. Based on the three-dimensional morphological features and high-order mathematical features, obtain related features through a preset correlation model.

[0056] S520. Generate dynamic fusion weights based on the related features.

[0057] S530. Based on the fusion weights, fuse the three-dimensional morphological features and high-order mathematical features to generate high-dimensional fused image features.

[0058] Among them, the preset association model generates associated features by constructing a joint embedding space to map three-dimensional morphological features and high-order mathematical features into the same space, then performing correlation statistics on the features, and setting a correlation threshold.

[0059] First, through the preset association model, associated features can be extracted for three-dimensional morphological features and high-order mathematical features, then the fusion weights can be dynamically adjusted for the associated features, and based on the generated fusion weights, the three-dimensional morphological features and high-order mathematical features can be fused to generate high-dimensional fused image features. This fusion method can fully explore the complementary information and correlation information between features compared to simply adding or splicing the two features, so as to achieve the effect of 1 + 1 > 2.

[0060] S600. Encode the associated clinical information and generate multi-dimensional feature labels in combination with the high-dimensional fused image features.

[0061] Among them, the associated clinical information includes off-table information and on-table information. The off-table information represents the physical information of the patient such as height, weight, and age; the on-table information represents the patient's lung function data and clinical symptom information.

[0062] Since the lesions of interstitial lung diseases are also related to the patient's own body posture and physical condition. For example, for UIP (usual interstitial pneumonia), the onset age is usually over 50 years old, and with the increase of age, the incidence rate gradually increases. While for NSIP (non-specific interstitial pneumonia), the onset age is relatively younger, mostly around 40 - 50 years old. In addition, height mainly affects lung volume, and weight affects lung load, both of which will have a certain impact on the lesions of interstitial diseases, and lung function is directly related to the lesions.

[0063] Therefore, in the embodiments of the present application, the associated clinical information, that is, the data information such as the patient's height, weight, age, and lung function, will also be incorporated into the classification consideration of the category. That is, after obtaining the high-dimensional fused image features, the associated clinical information will be encoded, and in combination with the high-dimensional fused image features, multi-dimensional feature labels will be generated, and the multi-dimensional feature labels will be used as the final classification target.

[0064] Specifically, encoding the associated clinical information and generating multi-dimensional feature labels in combination with the high-dimensional fused image features, see Figure 5 , including the following steps: S610. Through the preset data analysis model, obtain the influence coefficients of the off-table information on various interstitial lung diseases and generate the first influence vector.

[0065] S620. Through the preset disease diagnosis library, obtain the matching degrees of the on-table information with various interstitial lung diseases and generate the second influence vector.

[0066] S630. Encode the clinical information based on the first influence vector and the second influence vector to generate clinical feature labels.

[0067] S640. Integrate the clinical feature labels with the high-dimensional fused image features to generate multi-dimensional feature labels.

[0068] Among them, the first influence vector and the second influence vector respectively represent the influence degrees of off-table information and on-table information on the determination of the current interstitial lung disease category.

[0069] The preset data analysis model is mainly a model generated by data analysis and modeling based on historical interstitial lung disease diagnosis data. With the help of this model, the influence coefficients of off-table information relative to various interstitial lung diseases can be calculated. The preset disease diagnosis library is a database constructed with historical interstitial lung disease diagnosis data. With the help of this diagnosis library, the matching degrees of the current on-table information with various interstitial lung diseases can be obtained.

[0070] In the embodiment of the present application, first, through the preset data analysis model, the influence coefficients of off-table information on various interstitial lung diseases can be obtained, and the first influence vector is generated.

[0071] Specifically, the on-table clinical information of the patient is input into the preset data analysis model. The model calculates the influence coefficients of the off-table information of this patient relative to various interstitial lung diseases, arranges these influence coefficients in a certain order, and generates the first influence vector, which is equivalent to making a possibility assessment of the interstitial lung disease category according to the off-table clinical information of the current patient, and the first influence vector is a data representation of the assessment result.

[0072] Then, through the preset disease diagnosis library, the matching degrees of on-table information with various interstitial lung diseases are obtained, and the second influence vector is generated.

[0073] Specifically, the on-table clinical information of the patient is matched with the data in the preset disease diagnosis library. By calculating similarity or other relevant indicators, the matching degrees of the current on-table information with various interstitial lung diseases are obtained, and these matching degrees are sorted by category to generate the second influence vector, which is equivalent to making a possibility assessment of another dimension of the interstitial lung disease category according to the on-table clinical information of the current patient, and the second influence vector is a data representation of another assessment result.

[0074] Then, based on the first influence vector and the second influence vector, the clinical information is encoded to generate clinical feature labels.

[0075] The first influence vector and the second influence vector here do not conflict with each other, so the two will be combined and jointly assist in the classification of the category of interstitial lung disease. Encoding clinical information means encoding off-table information and on-table information respectively, and the first influence vector and the second influence vector can be regarded as weight parameters for off-table information and on-table information respectively. By configuring the weight parameters for the encoded information, clinical feature labels are formed.

[0076] Finally, the clinical feature labels are integrated with the high-dimensional fused image features to generate multi-dimensional feature labels, that is, the clinical feature labels and the high-dimensional fused image features are integrated into a whole as the classification target, so as to carry out the classification of the category of interstitial lung disease. The multi-dimensional feature labels synthesize clinical information and image features, can more comprehensively describe the patient's condition and disease characteristics, and provide more valuable information for the category judgment of interstitial lung disease.

[0077] To integrate the generated clinical feature labels with the high-dimensional fused image features, methods such as splicing and weighted fusion can be used to combine the clinical feature labels and the high-dimensional fused image features into a multi-dimensional feature vector. For example, the clinical feature labels are used as one sub-vector, and the high-dimensional fused image features are used as another sub-vector for splicing operation to form a complete multi-dimensional feature vector.

[0078] S700. Based on the multi-dimensional feature labels, through a preset classification model, the classification result of interstitial lung disease is generated.

[0079] Among them, the preset classification model is an interstitial lung disease classification model obtained through training with the multi-dimensional feature labels as the classification target and a large amount of lung CT data and clinical diagnosis information as data.

[0080] Through the multi-dimensional feature labels generated according to the current lung CT image and associated clinical information, with the help of the preset classification model, the classification result of interstitial lung disease can be directly generated.

[0081] See Figure 6 , the upper part of the figure shows the most likely classification results of the lung CT images, as well as the confidence levels of each category (UIP, NSIP, OP, pUIP), and the lower part shows the regions where the lesion features are located.

[0082] In the embodiments of the present application, 7 conventional machine learning models are respectively used to generate the preset classification model, including SVM, random forest, MLP, XGBoost, KNN, logistic regression (LR), decision tree. In addition, an ensemble model is added, and classification experiments are carried out for each classification model. The experimental results are shown in Table 2.

[0083] Table 2: According to the data in the table, it can be seen as a whole that for the classification model constructed with multi-dimensional feature tags as classification targets, the classification accuracy (Accuracy) and precision (Precision) can basically reach over 85%, and the AUC is relatively high, indicating that the classification model has shown certain effectiveness and stability and has good potential for clinical auxiliary diagnosis. However, there is still a large room for improvement.

[0084] Because different interstitial lung diseases have differences in clinical manifestations, imaging features, pathophysiological mechanisms, etc., there is a problem of difficult model generalization.

[0085] Therefore, in the embodiments of the present application, after obtaining the classification result, the data diversity will be increased according to the classification result, and the preset classification model will be further optimized and trained to improve the generalization ability of the model.

[0086] Specifically, after generating the classification result of interstitial lung disease based on multi-dimensional feature tags through a preset classification model, the following steps are further included: S810: According to the classification result, obtain dynamic evolution features through a preset dynamic evolution model.

[0087] S820: Based on the dynamic evolution features, perform three-dimensional reconstruction on the current CT image to generate a new CT image.

[0088] S830: Add the new CT image to the preset training database.

[0089] The preset dynamic evolution model is a model constructed by performing big data analysis and modeling based on historical data to simulate a certain rule, because the disease changes of each type of interstitial lung disease from the early stage to the middle stage and then to the late stage conform to a certain rule. Through this model, the lesion state of the current interstitial lung disease in the next period of time or the next stage can be inferred for various types of interstitial lung diseases.

[0090] The dynamic evolution features represent the lesion features generated based on the subsequent change state of the current disease condition.

[0091] In order to enhance the data diversity of the preset classification model, after determining the classification result of the current pulmonary CT image, the data can be expanded by means of a preset dynamic evolution model, that is, the dynamic evolution features are obtained through the preset dynamic evolution model, and then the current pulmonary CT image is three-dimensionally reconstructed according to the dynamic evolution features, so that a new pulmonary CT image can be generated. Then, the new pulmonary CT image, combined with the current classification result and associated clinical information, is added to the preset training database, which can be used as subsequent expanded training data.

[0092] For example, if the current determination result based on the lung CT image is early UIP, the disease state of mid-stage UIP can be inferred, and according to the inferred situation, the dynamic evolution characteristics can be obtained. Based on the current CT image, a new CT image can be generated through three-dimensional reconstruction, thus adding a new piece of lung CT image data of the category of mid-stage UIP.

[0093] It should be noted that using the preset dynamic evolution model for inference cannot be reversed. For example, inferring the early disease state from the mid-stage or late stage is mainly because the development of the disease is an irreversible process, and the late disease state may be affected by various factors, resulting in information loss or confusion. For example, the pulmonary fibrosis changes that appear in the late stage may gradually develop from various early lesions, and it is difficult to accurately infer the specific early disease state based only on the late fibrosis manifestation. Moreover, different types of interstitial lung diseases may show some similar manifestations in the mid-stage or late stage, which also increases the difficulty of reverse inference.

[0094] In addition, it is also not possible to make cross-stage inferences. For example, it is not possible to directly infer the disease situation of late-stage UIP from the disease situation of early-stage UIP because there are too many variable factors, and cross-stage inferences are prone to large deviations.

[0095] Because it is considered that the evolution law of the disease state is greatly affected by the individual, that is, it has a large correlation with clinical information.

[0096] Therefore, in the embodiments of the present application, clinical information will also be added, and on the basis of the preset dynamic evolution model, clinical information will also be included in the consideration range to further determine the evolution trend of the disease state.

[0097] Specifically, according to the classification result, through the preset dynamic evolution model, the dynamic evolution characteristics are obtained, including the following steps: S811: According to the classification result, match the corresponding preset dynamic evolution model from the preset dynamic evolution database.

[0098] S812: Based on the current CT image and the target features, generate variable features through the preset dynamic evolution model.

[0099] S813: Based on the associated clinical information, correct the variable features to generate dynamic evolution features.

[0100] Because different categories of interstitial lung diseases have different lesion evolution laws, there are multiple preset dynamic evolution models, which are stored in the preset dynamic evolution database. Therefore, first, according to the current classification result, the corresponding preset dynamic evolution model is matched from the preset dynamic evolution database, and then based on the current CT image and the target features, variable features are generated through the preset dynamic evolution model.

[0101] In addition to the possible changes in the current target features, the variable features here may also include newly added target features, that is, the variable features are equivalent to new pathological features.

[0102] Considering that the patient's clinical information will also affect the changes in the disease condition, after obtaining the variable features, the variable features will be corrected based on the associated clinical information. For example, at different age stages, the elasticity of lung tissue is different, and the corresponding change trends and amplitudes of pathological features such as honeycombing and reticulation will also vary. Therefore, after obtaining the variable features, the associated clinical information can be used to judge the rationality of the variable features, and the unreasonable variable features can be corrected. The variable features obtained after correction are the dynamic evolution features.

[0103] After generating the newly added lung CT images for the current lung CT image and classification result through the dynamic evolution model, the newly added lung CT images will be added to the preset database as the subsequent expanded training dataset. When the subsequent expanded dataset reaches a certain amount, iterative training can be performed on the basis of the current preset classification model to optimize the model.

[0104] Specifically, after adding the newly added CT images to the preset training database, the following steps are further included: S910. Judge whether the data volume of the newly added CT images in the current preset training database reaches the preset update threshold.

[0105] S920. If so, perform iterative update on the preset classification model based on the current preset training database to generate a new preset classification model.

[0106] When newly added lung CT images are stored in the preset training database, it will be determined whether the data volume of the newly added CT images in the current preset training database reaches the preset update threshold. If it reaches the preset update threshold, the preset training database can be used as the expanded training dataset to perform iterative update on the preset classification model to generate a new preset classification model, that is, a further optimized classification model. In this way, the model can be continuously improved to enhance the generalization ability of the classification model.

[0107] The embodiment of the present application also provides an interstitial lung disease classification system based on multi-dimensional feature labels. Refer to Figure 7 , this system includes: a data acquisition module 101, a region segmentation module 102, a feature detection module 103, a multi-dimensional feature fusion module 104, and a classification result generation module.

[0108] Among them, the data acquisition module 101 is used to acquire lung CT image information, and the CT image information includes CT images and associated clinical information.

[0109] The region segmentation module 102 is configured to perform region segmentation on the CT image through a preset three-dimensional model to obtain the three-dimensional spatial information of each region.

[0110] The feature detection module 103 is configured to perform target feature detection on the CT image based on the three-dimensional spatial information of each region to obtain the three-dimensional morphological features of the target features.

[0111] The multi-dimensional feature fusion module 104 is configured to perform deep feature extraction and mathematical calculation on the target features to obtain high-order mathematical features, fuse the three-dimensional morphological features and the high-order mathematical features to generate high-dimensional fused image features, encode the associated clinical information, and combine the high-dimensional fused image features to generate multi-dimensional feature labels.

[0112] The classification result generation module 105 is configured to generate the classification result of interstitial lung disease through a preset classification model based on the multi-dimensional feature labels.

[0113] In the embodiment of the present application, the data acquisition module 101 is specifically configured to acquire pulmonary CT image information, where the CT image information includes a CT image and associated clinical information, and perform corresponding preprocessing on the CT image.

[0114] The region segmentation module 102 is specifically configured to perform region segmentation on the CT image acquired by the data acquisition module 101 through a preset three-dimensional model to obtain the three-dimensional spatial information of the left lung, right lung, and regions of the left upper lobe, left lower lobe, right upper lobe, right middle lobe, and right lower lobe of the lung.

[0115] The feature detection module 103 is specifically configured to perform target feature detection on the CT image according to the three-dimensional spatial information of each region acquired by the region segmentation module 102 to obtain the three-dimensional morphological features of the target features, that is, lesion features, including morphological feature information and spatial occupancy information.

[0116] The multi-dimensional feature fusion module 104 is specifically configured to perform deep feature extraction and mathematical calculation on the target features acquired by the feature detection module 103 to obtain high-order mathematical features, fuse the three-dimensional morphological features and the high-order mathematical features to generate high-dimensional fused image features, encode the associated clinical information, and combine the high-dimensional fused image features to generate multi-dimensional feature labels.

[0117] The classification result generation module 105 is specifically configured to generate the classification result of interstitial lung disease through a preset classification model according to the multi-dimensional feature labels acquired by the multi-dimensional feature fusion module 104.

[0118] The embodiment of the present application further provides a computer-readable storage medium storing a computer program that can be loaded and executed by a processor to perform any one of the above methods for classifying interstitial lung disease based on multi-dimensional feature labels.

[0119] The embodiments of the specific implementation manners are all preferred embodiments of the present application, and do not limit the protection scope of the present application. Therefore, all equivalent changes made according to the principles of the present application shall be covered within the protection scope of the present application.

Claims

1. A classification method for interstitial lung disease based on multi-dimensional feature tags, characterized in that, Comprising: Obtain pulmonary CT image information, where the CT image information includes CT images and associated clinical information; Perform regional segmentation on the CT images through a preset three-dimensional model to obtain the three-dimensional spatial information of each region; Based on the three-dimensional spatial information of each region, perform target feature detection on the CT images to obtain the three-dimensional morphological features of the target features; Perform deep feature extraction and mathematical calculations on the target features to obtain high-order mathematical features; Fuse the three-dimensional morphological features and high-order mathematical features to generate high-dimensional fused image features; Encode the associated clinical information and combine it with the high-dimensional fused image features to generate multi-dimensional feature labels; Based on the multi-dimensional feature labels, generate a classification result of interstitial lung disease through a preset classification model.

2. The interstitial lung disease classification method based on multi-dimensional feature tags according to claim 1, wherein The three-dimensional morphological features include morphological feature information and spatial occupancy information. The performing target feature detection on the CT images based on the three-dimensional spatial information of each region to obtain the three-dimensional morphological features of the target features includes: Perform target feature detection on the CT images to obtain the morphological feature information and three-dimensional spatial information of the target features; Based on the three-dimensional spatial information of each region, match the spatial position information of the target feature in the associated region; Based on the three-dimensional spatial information of each region and the target feature, calculate and obtain the volume of each region and the volume of the target feature respectively; Based on the volume of each region and the volume of the target feature, calculate and obtain the spatial occupancy information of the target feature through the spatial position information of the target feature in the associated region.

3. A classification method for interstitial lung diseases based on multi-dimensional feature tags according to claim 1, characterized in that, The performing deep feature extraction on the target features to obtain high-order mathematical features includes: Based on the target features, extract deep features through a preset deep neural network; Perform mathematical calculations on the deep features to obtain mathematical features; Fuse the deep features and the mathematical features to generate high-order mathematical features.

4. A classification method for interstitial lung disease based on multi-dimensional feature tags according to claim 1, characterized in that, The fusing the three-dimensional morphological features and the high-order mathematical features to generate high-dimensional fused image features includes: Based on the three-dimensional morphological features and high-order mathematical features, obtain associated features through a preset association model; Generate a fusion weight based on the associated features; Based on the fusion weight, fuse the three-dimensional morphological features and high-order mathematical features to generate high-dimensional fused image features.

5. A classification method for interstitial lung disease based on multi-dimensional feature tags according to claim 1, characterized in that, The associated clinical information includes off-table information and in-table information. The encoding the associated clinical information and combining it with the high-dimensional fused image features to generate multi-dimensional feature labels includes: Through a preset data analysis model, obtain the influence coefficients of the off-table information on various types of interstitial lung diseases and generate a first influence vector; Through a preset disease diagnosis library, obtain the matching degrees of the in-table information with various types of interstitial lung diseases and generate a second influence vector; Based on the first influence vector and the second influence vector, encode the clinical information to generate clinical feature labels; Integrate the clinical feature labels with the high-dimensional fused image features to generate multi-dimensional feature labels.

6. A classification method for interstitial lung disease based on multi-dimensional feature tags according to claim 1, characterized in that After generating the classification result of interstitial lung disease based on the multi-dimensional feature labels through a preset classification model, it further includes: According to the classification result, obtain dynamic evolution features through a preset dynamic evolution model; Based on the dynamic evolution features, perform three-dimensional reconstruction on the current CT image to generate a new CT image; Add the newly added CT images to the preset training database.

7. A method for classifying interstitial lung diseases based on multi-dimensional feature tags according to claim 6, characterized in that, According to the classification result, obtaining dynamic evolution features through a preset dynamic evolution model, including: According to the classification result, match the corresponding preset dynamic evolution model from the preset dynamic evolution database; Based on the current CT image and target features, generate variable features through the preset dynamic evolution model; Based on the associated clinical information, correct the variable features to generate dynamic evolution features.

8. The interstitial lung disease classification method based on multi-dimensional feature tags according to claim 6, characterized in that, After adding the newly added CT images to the preset training database, it further includes: Judge whether the data volume of the newly added CT images in the current preset training database reaches the preset update threshold; If so, based on the current preset training database, iteratively update the preset classification model to generate a new preset classification model.

9. An interstitial lung disease classification system based on multi-dimensional feature tags, characterized in that, Including: A data acquisition module (101) for acquiring pulmonary CT image information, where the CT image information includes CT images and associated clinical information; A region segmentation module (102) for performing region segmentation on the CT image through a preset three-dimensional model to obtain the three-dimensional spatial information of each region; A feature detection module (103) for performing target feature detection on the CT image based on the three-dimensional spatial information of each region to obtain the three-dimensional morphological features of the target features; A multi-dimensional feature fusion module (104) for performing deep feature extraction and mathematical calculations on the target features to obtain high-order mathematical features, fusing the three-dimensional morphological features and high-order mathematical features to generate high-dimensional fusion image features, encoding the associated clinical information, and combining the high-dimensional fusion image features to generate multi-dimensional feature labels; A classification result generation module (105) for generating a classification result of interstitial lung disease based on the multi-dimensional feature labels through a preset classification model.

10. A computer-readable storage medium storing a computer program that can be loaded and executed by a processor to perform a method for classifying interstitial lung disease based on multi-dimensional feature labels as described in any one of claims 1 to 8.

Images