Method, device, medium, and program product for interstitial lung disease prediction based on residual network and multiple instances

Abstract

This invention provides a method, device, medium, and program product for predicting interstitial lung disease (ILD) based on residual networks and multiple instances. It also provides a method, device, medium, and program product for constructing an ILD prediction model based on residual networks and multiple instances, relating to the field of intelligent healthcare. The method of this invention constructs a prediction model for common ILD subtypes by aggregating residual networks and histogram-based multiple instances in a convolutional neural network. Through a deep learning model, it identifies HP, NSIP, OP, and UIP in HRCT images, reliably identifying the corresponding ILD subtypes in HRCT images with an accuracy level comparable to that of experienced ILD specialists.

Description

Technical Field

[0001] This invention relates to the field of intelligent healthcare, and more specifically, to a method, device, medium, and program product for predicting interstitial lung disease based on residual networks and multiple instances. Background Technology

[0002] Interstitial lung disease (ILD) is a group of life-threatening, heterogeneous diseases characterized by progressive inflammation and / or fibrosis of the lungs, which can eventually lead to respiratory failure. Clinically, a major challenge with ILD is the difficulty in making a rapid and accurate diagnosis, often resulting in delays in treatment intervention. High-resolution computed tomography (HRCT) is one of the key techniques for diagnosing and assessing ILD. Different imaging patterns in patients can lead to different prognoses; for example, patients with confirmed interstitial pneumonia (UIP) have a higher mortality rate. However, the classification of ILD on HRCT is often influenced by observer subjectivity and heavily relies on the knowledge and experience of the radiologist. This variability can lead to inconsistencies in diagnosis, and diagnostic delays can severely impact patient prognosis, as early identification and treatment are crucial for slowing disease progression and improving quality of life.

[0003] Recent advances in artificial intelligence (AI) have shown promise in overcoming the limitations of human interpretation, improving diagnostic accuracy and consistency by providing objective and automated diagnostic assessments of lung images. Deep learning algorithms, particularly convolutional neural networks (CNNs), have demonstrated impressive capabilities in medical imaging tasks. However, the performance of AI models can vary significantly across different patient populations and clinical settings. Currently, there is an urgent need to develop specialized AI diagnostic models for ILD subtypes prevalent in Asian populations. Summary of the Invention

[0004] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention provides a method, device, medium, and program product for predicting interstitial lung disease based on residual networks and multiple instances. The method of this invention constructs a predictive model for common subtypes of interstitial lung disease by aggregating residual networks in convolutional neural networks and histogram-based multiple instances. Through a deep learning model, it identifies HP, NSIP, OP, and UIP in HRCT images, reliably identifying the corresponding interstitial pneumonia subtypes in HRCT images with an accuracy reaching the level of experienced interstitial lung disease experts.

[0005] The first aspect of this application discloses a method for constructing a predictive model for interstitial lung disease, the method comprising:

[0006] 101. Obtain chest CT images and corresponding diagnostic labeling of training set samples; the CT images include images of at least two of the following pneumonia types: allergic pneumonia, nonspecific interstitial pneumonia, organizing pneumonia, and common interstitial pneumonia; the CT images are 2D slice images;

[0007] 102. The 2D slice image is input into a supervised convolutional neural network SPAIDNet framework for training, and the first probability score of a single 2D slice image belonging to each pneumonia type is calculated.

[0008] 103, set the first probability scores of all 2D slice images in a single sample; count the number of 2D slice images corresponding to the same first probability score;

[0009] 104. Input the number of 2D slice images, the first probability score of a single 2D slice image and its corresponding pneumonia type into the classifier to obtain the predicted classification result. Compare the result with the corresponding diagnostic label and optimize the model based on the comparison result to obtain the constructed prediction model.

[0010] In some embodiments, the method further includes processing the number of decimal places of the first probability fraction in 102 to obtain a second probability fraction with the same number of decimal places; the second probability fraction is in the range between the minimum and maximum values.

[0011] Optionally, the number of decimal places is at least one, and preferably two;

[0012] Optionally, the standard for processing the decimal number includes any of the following: rounding, or directly discarding the Nth decimal digit, where N is a natural number greater than or equal to 1.

[0013] In some embodiments, between 103 and 104, the method further includes: normalizing the number of 2D slice images, the first probability score of a single 2D slice image and its corresponding pneumonia type, and inputting the normalized features into a classifier;

[0014] Optionally, the normalization method includes: dividing the difference between each true value of a feature and the mean value of that feature by the standard deviation of that feature;

[0015] Optionally, the CT image is an HRCT image.

[0016] In some embodiments, the supervised convolutional neural network SPAIDNet framework includes a pre-trained ResNet-18, a max pooling module, and a fully connected layer with a softmax activation function.

[0017] Optionally, the ResNet-18 is an 18-layer residual framework deep learning model; the max pooling module performs feature aggregation; the combined features are input into the fully connected layer, and the softmax activation function is used to generate the probability score of a single sample having each type of pneumonia (HP, NSIP, OP, UIP);

[0018] Optionally, the supervised convolutional neural network SPAIDNet framework is trained for M epochs using the SGD optimizer, with an initial learning rate of a and a mini-batch size of b.

[0019] Optionally, in each epoch, the algorithm will input all samples into the model in a set order to perform forward propagation, calculate loss, backpropagation, and parameter update;

[0020] Optionally, for disease types with small sample sizes, instance-balanced sampling can be used to mitigate the impact of class imbalance, and class-balanced loss can be selected as the loss function to train the model.

[0021] In some embodiments, the classifier includes any one or more of the following: K-nearest neighbors, decision tree, Naive Bayes, logistic regression, support vector machine, random forest, gradient boosting tree, and multilayer perceptron;

[0022] Optionally, between steps 101 and 102, the method further includes: performing preprocessing on the CT image, including resampling, standardization, and cropping, to obtain a preprocessed 2D slice image.

[0023] A second aspect of this application discloses a method for predicting interstitial lung disease based on residual networks and multiple instances, the method comprising:

[0024] 201. Acquire the subject's CT image to be identified;

[0025] 202, The CT image to be identified is input into the prediction model disclosed in the first aspect of this application to obtain the auxiliary prediction classification result;

[0026] Optionally, the CT image to be identified is a chest CT image;

[0027] Optionally, the CT image to be identified is an HRCT image.

[0028] In some embodiments, the auxiliary prediction classification results include any one or more of the following: allergic pneumonia (HP), nonspecific interstitial pneumonia (NSIP), organizing pneumonia (OP), and common interstitial pneumonia (UIP).

[0029] A third aspect of this application discloses a computer device, the device comprising: a memory and a processor; the memory being used to store a computer program; and the processor executing the computer program to implement the steps of the above-described method.

[0030] The fourth aspect of this application discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0031] The fifth aspect of this application discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0032] This application has the following beneficial effects:

[0033] 1. This application innovatively discloses a method for constructing a prediction model for interstitial lung disease. This method aggregates residual networks and histogram-based multiple instances within a convolutional neural network (CNN). It trains a supervised CNN framework (SPAIDNet) on a dataset of HRCT images containing HP, NSIP, OP, and UIP to obtain a trained recognition model. Then, histogram-based likelihood aggregation is used to predict the class probabilities across all 2D layers of HRCT images, yielding a 3D classification result. In clinical applications, the trained model is used to identify real data to determine the classification of the corresponding HRCT image into HP, NSIP, OP, or UIP. This method reliably identifies the corresponding interstitial pneumonia subtype in HRCT images, achieving an accuracy comparable to that of experienced interstitial lung disease specialists. The method disclosed in this invention effectively overcomes the problem of inaccurate diagnostic conclusions in the prior art when diagnosing common subtypes of interstitial lung disease based on CT images by using histogram likelihood aggregation to predict the class probabilities of all 2D layers of HRCT, and then using a classifier for training. Specifically, existing methods require diagnosing each section of the CT image, which is decomposed into 2D images, and then making a final diagnosis based on the section results. However, assuming there are 200 images, if the diagnosis result obtained from 100 images is result A, and the diagnosis result obtained from the other 100 images is result B, it is difficult to obtain a more accurate conclusion.

[0034] 2. This application innovatively adopts instance-balanced sampling to mitigate the impact of class imbalance based on the sample size of the four pneumonia disease types, and selects class-balanced loss as the loss function to train the model, so as to ensure the accuracy of the prediction results of HP, NSIP, OP and UIP by the application end. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a schematic diagram of the method flow provided in the first aspect of the present invention;

[0037] Figure 2 This is a schematic diagram of the method flow provided in the second aspect of the present invention;

[0038] Figure 3 This is a schematic diagram of the system for constructing the interstitial lung disease prediction model provided in an embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram of the interstitial lung disease prediction system based on residual networks and multiple instances provided in an embodiment of the present invention;

[0040] Figure 5 This is a schematic diagram of a computer device provided in an embodiment of the present invention;

[0041] Figure 6 This is a schematic diagram of the architecture of an exemplary computing device provided in an embodiment of the present invention;

[0042] Figure 7 This is a schematic diagram of the storage medium provided in an embodiment of the present invention;

[0043] Figure 8 This is a block diagram of an intelligent classification system for common subtypes of interstitial lung disease based on residual networks and histogram multi-instance aggregation, provided in an embodiment of the present invention.

[0044] Figure 9 This is a classification block diagram of the SPAIDNet model provided in an embodiment of the present invention;

[0045] Figure 10 This is a model performance diagram provided in an embodiment of the present invention; wherein, Figure 10 A represents the ROC curve of the internal validation set. Figure 10 B is the classification confusion matrix of the internal validation set. Figure 10 C represents the ROC curve of external test set 1. Figure 10 D is the classification confusion matrix of external test set 1. Figure 10 E represents the ROC curve for external test set 2. Figure 10 F is the classification confusion matrix of the external test set 2;

[0046] Figure 11These are representative Grad-CAM heatmaps of the HP, NSIP, OP, and UIP models provided in this embodiment of the invention. Figure 11 A represents the original HP HRCT image (middle), the corresponding attention heatmap for HP (left), and the Merge image obtained by pairing the original HP HRCT image with the corresponding attention heatmap. Figure 11 B represents the original HRCT image of NSIP (middle), the corresponding attention heatmap of NSIP (left), and the Merge image after pairing the original HRCT image of NSIP with the corresponding attention heatmap. Figure 11 C represents the original HRCT image of the OP (middle), the attention heatmap corresponding to the OP (left), and the Merge image after pairing the original HRCT image of the OP with the corresponding attention heatmap. Figure 11 D represents the original HRCT image of UIP (middle), the corresponding attention heatmap of UIP (left), and the Merge image after pairing the original HRCT image of UIP with the corresponding attention heatmap;

[0047] Figure 12 These are the ROC curves of different diseases identified using the SPAIDNet model and those identified manually, provided in the embodiments of the present invention. Figure 12 A represents the ROC curves for identifying junior radiologists in different cohorts. Figure 12 B represents the ROC curves for identification by junior radiologists in different cohorts with AI assistance. Figure 12 C represents the ROC curves used by senior thoracic radiologists in different cohorts. Figure 12 D represents the ROC curves of the identification of senior thoracic radiologists in different cohorts with AI assistance. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0049] In some of the processes described in the specification, claims, and accompanying drawings of this invention, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Figure 2 This is a schematic flowchart of a method for predicting interstitial lung disease based on residual networks and multiple instances, provided by an embodiment of the present invention. Specifically, the method includes the following steps:

[0052] 201: Acquire the subject's CT image to be identified;

[0053] In some embodiments, the terms “subject” or “test subject” or “sample to be tested” as used herein refer to any animal (e.g., a mammal), including but not limited to humans, non-human primates, rodents, etc., which will become the recipient of a particular treatment. Generally, the terms “subject” and “patient” are used interchangeably herein when referring to human subjects. Preferably, the subject is a human. In some embodiments, the sample to be tested is a patient clinically used for prognostic assessment.

[0054] 202: Input the CT image to be identified into the prediction model disclosed in the first aspect of this application to obtain the auxiliary prediction classification result;

[0055] Optionally, the CT image to be identified is a chest CT image; alternatively, the CT image to be identified is a HRCT image.

[0056] In some embodiments, the auxiliary prediction classification results include any one or more of the following: allergic pneumonia (HP), nonspecific interstitial pneumonia (NSIP), organizing pneumonia (OP), and common interstitial pneumonia (UIP).

[0057] In some embodiments, the auxiliary prediction results include, but are not limited to, paper or electronic reports. These results are obtained by intelligent machines based on the relevant data of the subjects and are intended only as a reference for medical personnel, not as the final diagnostic results of the subjects.

[0058] In some embodiments, such as Figure 1 As shown in the figure, a flowchart illustrating a method for constructing a predictive model for interstitial lung disease provided by an embodiment of the present invention is presented. The construction method includes:

[0059] 101. Obtain chest CT images and corresponding diagnostic labeling of training set samples; the CT images include images of at least two of the following pneumonia types: hypersensitivity pneumonitis (HP), nonspecific interstitial pneumonia (NSIP), organizing pneumonia (OP), and common interstitial pneumonia (UIP); the CT images are 2D slice images;

[0060] In some embodiments, between steps 101 and 102, the method further includes: performing preprocessing on the CT image including resampling, standardization, and cropping to obtain a preprocessed 2D slice image. Specifically, the preprocessing includes: resampling each HRCT image to 1*1*1mm and adjusting the image to a lung window (-1500 HU to -600 HU); scanning each HRCT image to perform lung image segmentation (without preserving the pulmonary artery and heart); using a window center of 100 and a window width of 700, performing window operations to adjust the lung region to 224*224; and performing min-max normalization on the resized lung region.

[0061] 102. The 2D slice image is input into a supervised convolutional neural network SPAIDNet framework for training, and the first probability score of a single 2D slice image belonging to each pneumonia type is calculated.

[0062] In some embodiments, the method further includes processing the number of decimal places of the first probability fraction in 102 to obtain a second probability fraction with the same number of decimal places; the second probability fraction is in the range between the minimum and maximum values; optionally, the number of decimal places is at least 1, preferably 2; optionally, the standard for processing the number of decimal places includes any of the following: rounding, directly discarding the Nth decimal place, where N is a natural number greater than or equal to 1.

[0063] In some embodiments, the supervised convolutional neural network SPAIDNet framework includes a pre-trained ResNet-18, a max pooling module, and a fully connected layer with a softmax activation function.

[0064] Optionally, the ResNet-18 is an 18-layer residual framework deep learning model, whose weights are pre-trained using ImageNet and contain 10 million trainable parameters to extract instance features; the max pooling module performs feature aggregation; the combined features are input into the fully connected layer, and the softmax activation function is used to generate the probability score of a single sample having each type of pneumonia (HP, NSIP, OP, UIP), and the fully connected layer is initialized using the Kaiming method;

[0065] Optionally, the supervised convolutional neural network SPAIDNet framework is trained for M epochs using the SGD optimizer, with an initial learning rate of a and a mini-batch size of b, where M is a natural number greater than 1, preferably 50; a is 0.001; and b is 32.

[0066] Optionally, in each epoch, the algorithm will input all samples into the model in a set order for forward propagation, loss calculation, backpropagation, and parameter update; optionally, for disease types with small sample sizes, instance-balanced sampling is used to mitigate the impact of class imbalance, and class-balanced loss is selected as the loss function to train the model.

[0067] 103, set the first probability scores of all 2D slice images in a single sample; count the number of 2D slice images corresponding to the same first probability score;

[0068] In some embodiments, between 103 and 104, the method further includes: normalizing the number of 2D slice images, the first probability score of a single 2D slice image and its corresponding pneumonia type, and inputting the normalized features into a classifier; optionally, the normalization method includes: dividing the difference between each true value of a feature and the average value of that feature by the standard deviation of that feature;

[0069] 104. Input the number of 2D slice images, the first probability score of a single 2D slice image and its corresponding pneumonia type into the classifier to obtain the predicted classification result. Compare the result with the corresponding diagnostic label and optimize the model based on the comparison result to obtain the constructed prediction model.

[0070] Optionally, the CT image is an HRCT image.

[0071] In some embodiments, the classifier includes any one or more of the following: K-nearest neighbors, decision tree, Naive Bayes, logistic regression, support vector machine, random forest, gradient boosting tree, and multilayer perceptron.

[0072] Figure 5 This is a schematic diagram of a computer device provided in an embodiment of the present invention, such as... Figure 5As shown, the device 2000 may include: one or more processors 2010 and one or more memories 2020; wherein the memories store computer-readable code that, when run by the one or more processors, can perform the methods described above.

[0073] The processor in this embodiment can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, operations, and logic block diagrams disclosed in this embodiment. The general-purpose processor can be a microprocessor or any conventional processor, and can be based on an x86 or ARM architecture.

[0074] In general, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0075] For example, the method or apparatus according to embodiments of this disclosure can also be used by means of Figure 6 The architecture of the computing device 3000 shown is used for implementation. For example... Figure 6 As shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage devices in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files used for processing and / or communication of the methods provided in this disclosure, as well as program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Figure 6 The architecture shown is merely exemplary and can be omitted as needed when implementing different devices. Figure 6 One or more components in the computing device shown.

[0076] This invention also includes a computer-readable storage medium, such as... Figure 7The diagram illustrates a storage medium 4000 provided in an embodiment of the present invention. The computer storage medium 4020 stores computer-readable instructions 4010. When the computer-readable instructions 4010 are executed by a processor, the method described above according to embodiments of the present disclosure can be performed. The computer-readable storage medium in the embodiments of the present disclosure may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Synchronous Link Dynamic Random Access Memory (SLDRAM), and Direct Memory Bus Random Access Memory (DR RAM). It should be noted that the memory used in the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0077] This disclosure also provides a computer program product or system, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0078] In some embodiments, this embodiment also discloses a system for constructing a predictive model for interstitial lung disease, such as... Figure 3 As shown, the system includes:

[0079] The first acquisition module 301 is used or configured to acquire chest CT images and corresponding diagnostic result labels of training set samples; the CT images include images of at least two of the following pneumonia types: allergic pneumonia, nonspecific interstitial pneumonia, organizing pneumonia, and common interstitial pneumonia; the CT images are 2D slice images;

[0080] The probability score calculation module 302 is used or configured to input the 2D slice image into a supervised convolutional neural network SPAIDNet framework for training, and calculate the first probability score of a single 2D slice image belonging to each pneumonia type.

[0081] The probability score processing module 303 is used or configured to collect the first probability scores of all 2D slice images in a single sample; and count the number of 2D slice images corresponding to the same first probability score.

[0082] The prediction model training module 304 is used or configured to input the number of 2D slice images, the first probability score of a single 2D slice image and its corresponding pneumonia type into a classifier to obtain a prediction classification result, compare it with the corresponding diagnosis result label, optimize the model based on the comparison result, and obtain the constructed prediction model.

[0083] In some embodiments, this embodiment also discloses an interstitial lung disease prediction system based on residual networks and multiple instances, such as... Figure 4 As shown, the system includes:

[0084] The second acquisition module 401 is used or configured to acquire the subject's CT image to be identified;

[0085] The classification result output module 402 is used or configured to input the CT image to be identified into the prediction model disclosed in the first aspect of this application to obtain an auxiliary prediction classification result.

[0086] In some embodiments, this embodiment also discloses an intelligent classification and assessment system for common subtypes of interstitial lung disease based on residual networks and histogram multi-instance aggregation, such as... Figure 5 As shown, the system includes:

[0087] The data acquisition module acquires HRCT image datasets containing HP, NSIP, OP, and UIP for use in the deep learning model training module, internal testing module, and external validation module, as well as HRCT image data to be identified for the clinical application module.

[0088] The data preprocessing module resamples, standardizes, and trims the data collected by the data acquisition module to a suitable size to obtain data that meets the requirements of subsequent modules.

[0089] The deep learning model training module is implemented using the supervised convolutional neural network SPAIDNet framework. The SPAIDNet framework is trained with preprocessed data to obtain a trained recognition model. Then, histogram likelihood aggregation is used to obtain the class prediction probabilities of all 2D layers of HRCT to obtain the 3D classification results.

[0090] The internal testing module is used to perform internal testing on the trained recognition model based on the corresponding internal testing dataset.

[0091] The external validation module is used to perform recognition on the trained recognition model based on the corresponding external validation dataset, and then perform statistical analysis on the recognition results and human recognition results to verify the accuracy of the recognition model.

[0092] The clinical application module uses a recognition model to identify real data to be identified and determine whether the corresponding HRCT image belongs to HP, NSIP, OP, or UIP.

[0093] Furthermore, the intelligent classification and assessment system for common subtypes of interstitial lung disease based on residual networks and histogram multi-instance aggregation according to the present invention may also have the following additional technical features:

[0094] In some implementations, the data acquisition module acquires 5213 HRCT image datasets including HP, NSIP, OP, and UIP; of which 3585 are used as training datasets for the deep learning model training module, 932 are used as test datasets for the internal testing module, and 696 are used as verification test sets for the external verification module.

[0095] In some of these implementations, the inclusion criteria for the 5213 three-dimensional volumetric HRCT image dataset included: HP, NSIP, OP, and UIP that had been definitively diagnosed by multidisciplinary discussion.

[0096] In some implementations, the data preprocessing module processes the following: resamples each HRCT image to 1*1*1mm and adjusts it to the lung window (-1500 HU to -600 HU), then segments and crops all images of the lung region.

[0097] In some implementations, the window center is 100 and the window width is 700. Window operations are performed, and then the lung region is adjusted to 224*224. The resized lung region is then subjected to minimum-maximum normalization.

[0098] In some of these implementations, the supervised convolutional neural network SPAIDNet framework includes a pre-trained ResNet-18, a max-pooling module, and a fully connected layer with a softmax activation function.

[0099] The ResNet-18 contains 10 million trainable parameters to extract instance features;

[0100] The max pooling module uses max pooling operations to perform feature aggregation;

[0101] The fully connected layer is classified by generating probability scores using the softmax activation function.

[0102] In some implementations, the deep learning model training module receives the 2D image data processed by the data preprocessing module, obtains several 2D slices, forms a slice bag, and then inputs the slice bag into the supervised convolutional neural network SPAIDNet framework for model training.

[0103] In some implementations, the deep learning model training module uses a 10x cross-validation approach for model training; in each fold, 80% of the training data is used as the development set for model training, and the remaining 20% ​​is used as the tuning set for model selection; the model that achieves the best results on the tuning set is used for subsequent internal test sets and external validation.

[0104] In some implementations, the deep learning model training module uses an optimizer to train the entire network for 50 epochs with an initial learning rate of 0.001 and a mini-batch size of 32; it employs instance-balanced sampling to mitigate the impact of class imbalance and selects class-balanced loss as the loss function to train the model; and it uses a simplified LON platform method based on adjustment set loss to adjust the learning rate.

[0105] In some implementations, the system further includes a visualization module that uses gradient-weighted class activation mapping to visualize the model’s decision-making process.

[0106] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and specific examples and application scenarios.

[0107] Please see Figure 5 As shown, in some embodiments of the present invention, an intelligent classification and assessment system for common subtypes of interstitial lung disease based on residual networks and histogram multi-instance aggregation is provided. This system uses a deep learning model to identify HP, NSIP, OP, and UIP in HRCT images. The present invention collects several image data sets, divides them into three parts, and uses them for model training, internal testing, and external testing, respectively, thereby obtaining HP, NSIP, OP, and UIP identifications with satisfactory accuracy.

[0108] 1. Data Collection:

[0109] This invention collected HRCT images and corresponding diagnostic information from 5213 patients with HP, NSIP, OP, and UIP from three hospitals (including 3433 male patients, mean age = 61.735 ± 10.841 years). HRCT image data from the first hospital between July 2016 and June 2023 were used as the training set and internal test set; HRCT image data from the second and third hospitals between July 2016 and June 2023 were used as the external validation test set. The inclusion criteria for the dataset were as follows: a confirmed diagnosis of one of HP, NSIP, OP, or UIP by multidisciplinary discussion; age over 18 years; and signed informed consent. HRCTs with poor image quality or severe artifacts were excluded. HRCTs without digital imaging or communication in medical format (DICOM) were excluded. Patients diagnosed with other types of interstitial lung disease, including sarcoidosis, connective tissue disease-associated interstitial pneumonia, and drug-associated interstitial pneumonia, were excluded.

[0110] II. HRCT Scanning Protocol:

[0111] All patients underwent high-resolution computed tomography (HRCT) scans on multi-slice detector CT systems, including GE Healthcare's LightSpeed ​​VCT / 64 (Chicago, Illinois, USA), Toshiba's Aquilion ONETSX-301C / 320 (Tokyo, Japan), Philips' iCT / 256 (Amsterdam, Netherlands), and Siemens' FLASH dual-source CT (Erlangen, Germany). Scans were performed in the supine position and completed in a single breath-hold scan.

[0112] Scanning and reconstruction parameters strictly adhered to prescribed CT standards. Tube voltage was set within the range of 100–120 kV, tube current within the range of 100–300 mAs, and gantry rotation speed was 0.8 seconds. Images were progressively reconstructed with slice thicknesses ranging from 0.625–1.25 mm and a stage movement speed of 39.37 mm / s.

[0113] 3. Image Preprocessing:

[0114] Several preprocessing steps were performed on all HRCT image datasets in DICOM format. Specifically, each HRCT image was first resampled to a voxel representing 1*1*1 mm, then the image was adjusted to a lung window (-1500 HU to -600 HU). Next, lung image segmentation was performed on each HRCT scan using a deep learning method, which can be implemented using existing medical imaging software solutions. Then, the lung region was extracted from each CT scan using a 3D bounding box extracted with a corresponding segmentation mask, excluding the pulmonary artery and heart. Finally, windowing was performed with a window center of 100 and a window width of 700, and the lung region was resized to 224*224. The resized lung region was then subjected to min-max normalization.

[0115] Part 4: Deep Learning Models and Model Training

[0116] This invention employs the supervised convolutional neural network framework SPAIDNet for fine-grained classification. This framework (e.g., ...) Figure 9 As shown, a pre-trained ResNet-18 was used, an 18-layer residual framework deep learning model containing 10 million trainable parameters to extract instance features, and max pooling was employed for feature aggregation. The combined features were input into a fully connected layer, followed by a softmax activation function to generate probability scores for each type (HP, NSIP, OP, UIP). Since a patient has hundreds of corresponding 2D lung slices, ResNet-18 generates probability scores for each image. This invention further uses likelihood histogram aggregation to summarize all possible diagnostic results for a patient, forming 400 feature vectors. Finally, this invention inputs these feature vectors into a support vector machine to ultimately provide the diagnostic result for each patient.

[0117] Specifically, for HRCT images, the aforementioned preprocessing is first performed, and the processed image is segmented along the z-axis to obtain several image slices. Each obtained image slice is copied into a three-channel image as a two-dimensional instance. All instances are formed into a bag of slices. Each three-channel image in the bag is fed into the SPAIDNet model for training.

[0118] Model training employs a 10x cross-validation method. In each fold of this invention, patient-level training data is split, with approximately 80% of the training data used as the training set for model construction, and the remaining 20% ​​used as the tuning set for model selection. After training, the model that achieves optimal results on all tuning sets is applied to both the internal test set and the external validation set.

[0119] The ResNet-18 weights in the SPAIDNet model were pre-trained using ImageNet, and the classification layers (fully connected layers) were initialized using the Kaiming method. The entire network was trained for 50 epochs using the SGD optimizer, with an initial learning rate of 0.001 and a mini-batch size of 32. An epoch refers to the training process of going through all samples in the training dataset once (and only once). In one epoch, the training algorithm inputs all samples into the model in a predetermined order for forward propagation, loss calculation, backpropagation, and parameter updates. The learning rate is an important hyperparameter controlling the step size of weight updates; a smaller learning rate means the model updates weights more cautiously during training. The mini-batch size was set to 32, meaning each gradient update is based on a small number of samples. Considering the small sample size of the operating system (OP) in the training set, instance-balanced sampling was used to mitigate the impact of class imbalance, and class-balanced loss was chosen as the loss function for training the model. Data augmentation was performed using a toolkit called Batch Generator. During the training phase, the model's performance on the adjustment set was evaluated by accuracy.

[0120] In this invention, to aggregate instance-level diagnostic probabilities into comprehensive patient-level information, a histogram-based aggregation pipeline (SALH, Slide Aggregation via Likelihood Histograms) is designed. For each patient, the diagnostic probabilities of all their HRCT slices in four diagnostic categories (such as UIP, HP, etc.) are calculated, and a probability distribution histogram is constructed based on these probabilities.

[0121] Specifically, when a set of HRCT images containing 300 2D slices is input into the SPAIDNet model, the model provides four values ​​for each 2D slice, representing the probability that the slice belongs to one of the following categories: HP, NSIP, OP, and UIP. For each category, the probability data of all 300 slices are aggregated and rounded to two decimal places. If the probability is less than 0.005, it is rounded to 0.01; if it is greater than 0.995, it is rounded to 0.99. Therefore, for each category, the probability of each of the 300 2D slices belonging to that category will be a value between 0.00 and 0.99. The number of slices corresponding to each value is then counted, forming a probability distribution histogram. For each category, this histogram can be considered a 100-dimensional feature, representing the number of slices within a certain probability range for that specific category, such as the number of slices corresponding to a UIP diagnosis within a certain probability range (e.g., 0.850-0.855). Since this invention targets four common types of interstitial lung disease, a single HRCT image set processed through the aforementioned SPAIDNet and SALH pipeline generates 4 × 100 histogram features, totaling 400 features, representing the distribution of different diagnostic categories within different probability ranges. These features are then normalized and used to train a traditional machine learning classifier to further optimize diagnostic prediction performance. Normalization methods include, but are not limited to, dividing the difference between each true value of a feature and its mean by the standard deviation of that feature.

[0122] 5. Statistical Analysis:

[0123] This section focuses on the model's diagnostic performance for four disease groups, evaluating its performance in classifying HP, NSIP, OP, and UIP. Statistical analysis was performed using Python 3.7. The performance of the HP, NSIP, OP, and UIP classification was evaluated using the area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). A bootstrap test (1000 iterations) was used to estimate the 95% confidence interval (CI) for each evaluation indicator. Furthermore, the DeLong test was used to compare the model's AUC with the results of manual identification and judgment by radiologists of varying experience levels. Manual identification and judgment involved radiologists of different experience levels interpreting the displayed results of HRCT images.

[0124] The number of true positives, false positives, true negatives, and false negatives in classification performance, based on predictions from internal test and external validation sets, is also described in a 4×4 contingency table representing the confusion matrix.

[0125] VI: Analysis of Research Results

[0126] Table 1 shows the evaluation results of the classification model on the internal test set and the external validation set. According to the results, the SPAIDNet model of this invention has good correlations between its predictions and true labels and the four diseases. The AUCs for HP, NSIP, OP, and UIP are all close to 1.000, as shown in Table 1 below:

[0127]

[0128] ROC curve (i.e., the receiver's operating characteristic curve) as follows Figure 10 As shown in (A). Furthermore, there is a clear consistency between model predictions and ground facts in the internal validation set. Figure 10 (B) is the classification confusion matrix, which shows the number of true positive results, false positive results, true negative results, and false negative results predicted by the model.

[0129] To enhance the interpretability of the model, gradient-weighted class activation mapping (Grad-CAM) was employed. Figure 11 Representative Grad-CAM heatmaps for HP, NSIP, OP, and UIP models are shown. For each sample, the original HRCT image (center) is paired with the corresponding attention heatmap (left). Red highlights in the heatmaps show activation regions associated with the predicted class, with darker colors indicating stronger associations. Orange arrows indicate the actual location of the object in the original image as a lesion. The Grad-CAM heatmaps demonstrate that the model of this invention is able to attend to anomalies in the image during decision-making.

[0130] Model performance on the external validation set: The performance of the SPAIDNet model was further evaluated on the external validation set. A summary of the evaluation results can be found in [link to evaluation results]. Figure 10 And Table 2. As can be seen from the figures and tables, although the performance of the SPAIDNet model decreased on the external validation set, it still remained at a high level, as shown in Table 2 below:

[0131]

[0132] Furthermore, the ROC curves of the reader using the model of this invention and those of two manual identification processes are as follows: Figure 12As shown in Figures and Table 3, in external validation cohorts I and II, a junior radiologist with 3 years of experience achieved a diagnostic accuracy of 0.73 and a macro-mean AUC of 0.737 (0.691–0.784). However, their macro-mean sensitivity and specificity were only 0.573 and 0.513, respectively. Another senior thoracic radiologist with 10 years of experience achieved a higher diagnostic accuracy of 0.786 and a macro-mean AUC of 0.763 (0.713–0.812), but their macro-mean sensitivity and specificity remained limited, at 0.611 and 0.561, respectively. In the diagnosis of HP and OP, both radiologists showed significant improvements in diagnostic performance after using SPAIDNet for assistance. The junior radiologist's accuracy improved to 0.907, with a macro-mean AUC of 0.817 (0.779–0.855), sensitivity increased to 0.669, and specificity remained unchanged. For senior radiologists, the accuracy improved to 0.839, the macro-mean AUC was 0.787 (0.739-0.835), the sensitivity improved to 0.642, and the specificity remained unchanged. After incorporating AI into the results, the diagnostic abilities of both radiologists improved in the diagnosis of all four ILDs (OP: p<0.05, HP, NSIP, UIP: p<0.001), but no significant change was observed in the diagnosis of OP among senior thoracic radiologists. Furthermore, with AI assistance, junior radiologists outperformed senior radiologists in diagnosing HP, NSIP, and UIP, while there was no statistically significant difference in the diagnosis of OP. Ultimately, junior radiologists with AI assistance outperformed senior radiologists using AI in the diagnosis of NSIP and UIP, and also outperformed junior radiologists in the diagnosis of both NSIP and UIP (p<0.05). The SPAIDNet model outperformed experienced thoracic radiologists in diagnosing HP and OP, but achieved comparable performance in identifying NSIP and UIP. Table 3 is shown below:

[0133]

[0134] Junior: 5 years of experience as a radiologist; Senior: 15 years of experience as a respiratory imaging radiologist.

[0135] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0136] In general, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0137] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0138] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0139] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0140] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0141] The exemplary embodiments of this disclosure described in detail above are merely illustrative and not restrictive. Those skilled in the art will understand that various modifications and combinations can be made to these embodiments or their features without departing from the principles and spirit of this disclosure, and such modifications should fall within the scope of this disclosure.

Claims

1. A method for constructing a predictive model for interstitial lung disease, characterized in that, The method includes:

101. Obtain chest CT images and corresponding diagnostic result labels from the training set samples; the CT images include at least two of the following pneumonia types: HP, NSIP, OP, and UIP; the CT images are 2D slice images; 102. The 2D slice image is input into a supervised convolutional neural network SPAIDNet framework for training, and the first probability score of a single 2D slice image belonging to each pneumonia type is calculated.

103. Collect the first probability scores of all 2D slice images in a single sample; count the number of 2D slice images corresponding to the same first probability score, and form a probability distribution histogram based on the first probability score and the corresponding number of 2D slice images.

104. After performing likelihood aggregation on the probability distribution histogram, input it into the classifier to obtain the predicted classification result. Compare it with the corresponding diagnostic result label, optimize the model based on the comparison result, and obtain the constructed prediction model.

2. The method for constructing a predictive model for interstitial lung disease according to claim 1, characterized in that, The method further includes processing the number of decimals of the first probability fraction in 102 to obtain a second probability fraction with the same number of decimals; the second probability fraction is in the range between the minimum and maximum values.

3. The method for constructing a predictive model for interstitial lung disease according to claim 2, characterized in that, The minimum number of decimal places is one.

4. The method for constructing a predictive model for interstitial lung disease according to claim 2, characterized in that, The number of decimals is 2.

5. The method for constructing a predictive model for interstitial lung disease according to claim 2, characterized in that, The standard for processing the decimal number includes any of the following: rounding, or directly discarding the Nth decimal digit, where N is a natural number greater than or equal to 1.

6. The method for constructing a predictive model for interstitial lung disease according to claim 1, characterized in that, Between 103 and 104, the method further includes: normalizing the number of 2D slice images, the first probability score of a single 2D slice image and its corresponding pneumonia type, and inputting the normalized features into a classifier.

7. The method for constructing a predictive model for interstitial lung disease according to claim 6, characterized in that, The normalization method includes: dividing the difference between each true value of a feature and the average value of that feature by the standard deviation of that feature.

8. The method for constructing a predictive model for interstitial lung disease according to claim 1, characterized in that, The CT image is an HRCT image.

9. The method for constructing a predictive model for interstitial lung disease according to claim 1, characterized in that, The supervised convolutional neural network SPAIDNet framework includes a pre-trained ResNet-18, a max-pooling module, and a fully connected layer with a softmax activation function.

10. The method for constructing a predictive model for interstitial lung disease according to claim 9, characterized in that, The ResNet-18 is an 18-layer residual framework deep learning model; the max pooling module performs feature aggregation; the combined features are input into the fully connected layer, and the softmax activation function is used to generate the probability score of a single sample having each type of pneumonia.

11. The method for constructing a predictive model for interstitial lung disease according to claim 9, characterized in that, The supervised convolutional neural network SPAIDNet framework uses the SGD optimizer for training over M epochs, with an initial learning rate of a and a mini-batch size of b.

12. The method for constructing a predictive model for interstitial lung disease according to claim 11, characterized in that, In each epoch, the algorithm inputs all samples into the model in a set order to perform forward propagation, calculate loss, backpropagation, and parameter updates.

13. The method for constructing a predictive model for interstitial lung disease according to claim 11, characterized in that, For disease types with small sample sizes, instance-balanced sampling is used to mitigate the impact of class imbalance, and class-balanced loss is selected as the loss function to train the model.

14. The method for constructing a predictive model for interstitial lung disease according to claim 1, characterized in that, The classifier includes any one or more of the following: K-nearest neighbors, decision tree, Naive Bayes, logistic regression, support vector machine, random forest, gradient boosting tree, and multilayer perceptron.

15. The method for constructing a predictive model for interstitial lung disease according to claim 1, characterized in that, Between 101 and 102, the method further includes: performing preprocessing on the CT image, including resampling, standardization, and cropping, to obtain a preprocessed 2D slice image.

16. A method for predicting interstitial lung disease based on residual networks and multiple instances, characterized in that, The method includes:

201. Acquire the subject's CT image to be identified; 202. Input the CT image to be identified into the prediction model according to any one of claims 1-15 to obtain the auxiliary prediction classification result.

17. The interstitial lung disease prediction method based on residual networks and multiple instances according to claim 16, characterized in that, The CT image to be identified is a chest CT image.

18. The interstitial lung disease prediction method based on residual networks and multiple instances according to claim 16, characterized in that, The CT image to be identified is an HRCT image.

19. The interstitial lung disease prediction method based on residual networks and multiple instances according to claim 16, characterized in that, The auxiliary prediction classification results include any one or more of the following: allergic pneumonia, nonspecific interstitial pneumonia, organizing pneumonia, and common interstitial pneumonia.

20. A computer device, characterized in that, The device includes: a memory and a processor; the memory is used to store a computer program; the processor executes the computer program to implement the steps of the method according to any one of claims 1-19.

21. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, implements the steps of the method as described in any one of claims 1-19.

22. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-19.

Images