Method, apparatus, electronic device, and storage medium for processing lung images

Abstract

The present disclosure relates to a method, apparatus, electronic device, and storage medium for processing lung images, and relates to the technical field of chronic obstructive pulmonary disease (COPD) recognition. Among them, the method for processing lung images includes: obtaining a preset classification model, a lung image to be processed, and its corresponding lung region image and airway image; respectively obtaining corresponding first classification results, second classification results, and third classification results based on the preset classification model using the lung image to be processed, the lung region image, and the airway image; and performing COPD recognition based on the first classification result, the second classification result, and the third classification result. The embodiments of the present disclosure can achieve COPD recognition.

Description

Technical Field

[0001] The present disclosure relates to the technical field of chronic obstructive pulmonary disease (COPD) identification, and particularly to a method, apparatus, electronic device, and storage medium for processing lung images. Background Art

[0002] Chronic obstructive pulmonary disease (referred to as COPD) is a heterogeneous multi-system disease caused by continuous exposure to harmful particles, gases, or smoking. According to clinical characteristics, COPD can be classified into different features, such as emphysema and chronic bronchitis. Severe COPD can cause chronic morbidity and ultimately lead to death, and it will become the third leading cause of death in the world by 2030.

[0003] The diagnosis of COPD mainly relies on pulmonary function tests (PFTs), which use spirometry to evaluate airflow obstruction. According to the Global Initiative for Chronic Obstructive Lung Disease (GOLD), the diagnostic criteria for COPD are as follows: the ratio of FEV1 to FVC < 0.7 after inhaling a bronchodilator. PFT results only provide overall respiratory function parameters. Therefore, PFTs cannot be used to evaluate the regional morbidity and morphological abnormalities of COPD. In addition, the measurement accuracy of PFTs is limited by the cooperation of patients; the measurement process is very complex, and it is difficult for patients to understand and comply with the requirements put forward by doctors. Moreover, PFTs cannot intuitively provide detailed anatomical information and morphological changes, such as subtypes of emphysema and thickening of the bronchial wall.

[0004] Computed tomography (CT) can quantitatively describe the characteristics of COPD with high spatial resolution, but its contribution to diagnosis has not been fully exploited. Recently, significant progress has been made in CT imaging, especially high-resolution CT (HRCT), which has become an effective method for quantitative analysis of COPD, such as measuring gas trapping, the severity of emphysema, airflow obstruction, and small airway diseases. In addition, research on lung segmentation and airway segmentation has elucidated the potential advantages of using CT imaging to clearly describe the pulmonary anatomical structure. However, whether the segmentation of airways and lung regions can be beneficial for identifying COPD and improving its performance has not been reported.

[0005] In summary, there is an urgent need to provide an effective tool for early detection of COPD. Summary of the Invention

[0006] The present disclosure provides technical solutions for a method, apparatus, electronic device, and storage medium for processing lung images.

[0007] According to one aspect of the present disclosure, there is provided a method for processing a lung image, including:

[0008] Obtaining a preset classification model, a lung image to be processed, and its corresponding lung region image and airway image;

[0009] Based on the preset classification model respectively, use the lung image to be processed, the lung region image and the airway image to obtain the corresponding first classification result, second classification result and third classification result;

[0010] Based on the first classification result, the second classification result and the third classification result, identify chronic obstructive pulmonary disease.

[0011] Preferably, before obtaining the lung region image and the airway image corresponding to the lung image to be processed, it includes:

[0012] Obtain the lung image to be processed;

[0013] Perform lung region segmentation and airway extraction on the lung image to be processed respectively to obtain a lung region image and an airway image.

[0014] Preferably, the method of respectively obtaining the corresponding first classification result, second classification result and third classification result based on the preset classification model, using the lung image to be processed, the lung region image and the airway image includes:

[0015] Extract features from the lung image to be processed, the lung region image and the airway image respectively to obtain the corresponding first group of features, second group of features and third group of features;

[0016] Calculate the weight matrices corresponding to the first group of features, second group of features and third group of features respectively;

[0017] Fuse the first group of features, second group of features and third group of features and the corresponding weight matrices respectively to obtain the first classification feature, second classification feature and third classification feature;

[0018] Based on the first classification feature, the second classification feature and the third classification feature respectively, obtain the corresponding first classification result, second classification result and third classification result.

[0019] Preferably, the method of respectively fusing the first group of features, second group of features and third group of features and the corresponding weight matrices to obtain the first classification feature, second classification feature and third classification feature includes:

[0020] Perform matrix multiplication operations on the first group of features, second group of features and third group of features and the corresponding weight matrices respectively to obtain the first classification feature, second classification feature and third classification feature.

[0021] Preferably, before respectively performing feature extraction on the lung image to be processed, the lung region image, and the airway image to obtain corresponding first group of features, second group of features, and third group of features, respectively determining a first image corresponding to the lung image to be processed for feature extraction, and / or the lung region image, and / or the airway image, and / or a second image, and / or a third image, the determination method includes:

[0022] Delete the non-lung images in the lung image to be processed to obtain an image containing the lung; according to the obtained set number, extract the image containing the lung to obtain a first image corresponding to the feature extraction to be performed;

[0023] Perform three-dimensional reconstruction on the lung region image to obtain a three-dimensional lung region image; take pictures of the three-dimensional lung region image at multiple first set angles to obtain a second image corresponding to multiple first two-dimensional snapshots;

[0024] Perform three-dimensional reconstruction on the airway image to obtain a three-dimensional airway image; take pictures of the three-dimensional airway image at multiple second set angles to obtain a third image corresponding to multiple second two-dimensional snapshots.

[0025] Preferably, the method for respectively performing feature extraction on the lung image to be processed, the lung region image, and the airway image to obtain corresponding first group of features, second group of features, and third group of features includes:

[0026] Obtain a preset feature extraction model, train the preset feature extraction model to obtain a trained feature extraction model;

[0027] Based on the trained feature extraction model, respectively perform feature extraction on the lung image to be processed, the lung region image, and the airway image to obtain corresponding first group of features, second group of features, and third group of features.

[0028] Preferably, the method for identifying chronic obstructive pulmonary disease based on the first classification result, the second classification result, and the third classification result includes:

[0029] Respectively determine a first probability value, a second probability value, and a third probability value corresponding to the first classification result, the second classification result, and the third classification result;

[0030] Perform regression analysis on the first probability value, the second probability value, and the third probability value to identify chronic obstructive pulmonary disease;

[0031] Or,

[0032] Statistically analyze the first classification result, the second classification result, and the third classification result to obtain a first value corresponding to being determined as chronic obstructive pulmonary disease (COPD) and a second value corresponding to being determined as non-COPD;

[0033] If the first value is greater than the second value, determine it as COPD; otherwise, determine it as non-COPD.

[0034] According to one aspect of the present disclosure, there is provided a processing device for lung images, including:

[0035] An acquisition unit for acquiring a preset classification model, a lung image to be processed, and its corresponding lung region image and airway image;

[0036] A classification unit for respectively obtaining corresponding first, second, and third classification results based on the preset classification model by using the lung image to be processed, the lung region image, and the airway image;

[0037] An identification unit for identifying COPD based on the first classification result, the second classification result, and the third classification result.

[0038] According to one aspect of the present disclosure, there is provided an electronic device, including:

[0039] A processor;

[0040] A memory for storing instructions executable by the processor;

[0041] Wherein, the processor is configured to execute the above-mentioned processing method for lung images.

[0042] According to one aspect of the present disclosure, there is provided a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above-mentioned processing method for lung images is implemented.

[0043] In the embodiments of the present disclosure, respectively based on the preset classification model, corresponding first, second, and third classification results are obtained by using the lung image to be processed, the lung region image, and the airway image; and COPD is identified based on the first classification result, the second classification result, and the third classification result. It fully integrates the lung image intensity information and lung morphology information, solves the problem of the accuracy of COPD identification, and provides an effective tool for early detection of COPD.

[0044] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present disclosure.

[0045] According to the following detailed description of exemplary embodiments with reference to the accompanying drawings, other features and aspects of the present disclosure will become clear. Description of the Drawings

[0046] The drawings herein are incorporated into and constitute a part of this specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0047] Figure 1 A flowchart showing a method for processing a lung image according to an embodiment of the present disclosure;

[0048] Figure 2 A schematic diagram showing a network structure corresponding to a preset classification model according to an embodiment of the present disclosure;

[0049] Figure 3 A schematic diagram showing a second image and a third image corresponding to a lung region image and an airway image according to an embodiment of the present disclosure;

[0050] Figure 4 A schematic diagram showing a specific implementation of a method for processing a lung image according to an embodiment of the present disclosure;

[0051] Figure 5 A block diagram showing a processing apparatus for a lung image according to an embodiment of the present disclosure;

[0052] Figure 6 A block diagram of an electronic device 800 shown according to an exemplary embodiment;

[0053] Figure 7 A block diagram of an electronic device 1900 shown according to an exemplary embodiment. Detailed Description of the Embodiments

[0054] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the drawings. The same reference numerals in the drawings denote elements having the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0055] The special term "exemplary" herein means "serving as an example, embodiment, or illustrative". Any embodiment described as "exemplary" here does not necessarily have to be construed as superior to or better than other embodiments.

[0056] As used herein, the term "and / or" is merely a description of the associated relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" as used herein means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set composed of A, B, and C.

[0057] In addition, to better illustrate the present disclosure, numerous specific details are given in the following specific implementation manners. Those skilled in the art should understand that the present disclosure can also be implemented without certain specific details. In some instances, methods, means, elements, and circuits well-known to those skilled in the art are not described in detail so as to highlight the gist of the present disclosure.

[0058] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form combined embodiments without violating the principle logic. Due to space limitations, the present disclosure will not elaborate further.

[0059] In addition, the present disclosure also provides a lung image processing device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any one of the lung image processing methods provided by the present disclosure. The corresponding technical solutions and descriptions can be referred to the corresponding records in the method part and will not be elaborated further.

[0060] In the embodiments of the present disclosure and other possible embodiments, a total of 561 subjects (271 with chronic obstructive pulmonary disease (COPD) and 290 healthy controls (HC)) were obtained from the Affiliated Central Hospital of Shenyang Medical College (dataset 1, COPD: 170; HC: 190) and the Second Affiliated Hospital of Dalian Medical University (dataset 2, COPD: 101; HC: 100). This study was approved by the medical ethics committees of both hospitals. According to the Declaration of Helsinki (2000), all subjects gave informed consent. The diagnosis of all COPD subjects was made by experienced clinicians based on pulmonary function tests (PFT), and the PFT diagnostic criterion was FEV1 / FVC < 0.7 after inhalation of a bronchodilator. Dataset 2 also provided grading data for COPD subjects (stage I: 25, stage II: 43; stage III: 27; stage IV: 6).

[0061] Figure 1 A flowchart showing a method for processing a lung image according to an embodiment of the present disclosure is as Figure 1 shown. The method for processing the lung image includes: Step S101: Obtain a preset classification model, a lung image to be processed, and its corresponding lung region image and airway image; Step S102: Respectively based on the preset classification model, use the lung image to be processed, the lung region image, and the airway image to obtain corresponding first classification results, second classification results, and third classification results; Step S103: Based on the first classification result, the second classification result, and the third classification result, perform the identification of chronic obstructive pulmonary disease. Among them, the result of the COPD identification is having COPD or not having COPD. The embodiments of the present disclosure fully integrate the lung image intensity information and lung morphology information, solve the problem of the accuracy rate of COPD identification, and provide an effective tool for the early detection of COPD.

[0062] Step S101: Obtain a preset classification model, a lung image to be processed, and its corresponding lung region image and airway image.

[0063] In the embodiments of the present disclosure and other possible embodiments, the lung image may be one or several of CT images, MR images, or DR images. For another example, in the embodiments of the present disclosure and other possible embodiments, first obtain the lung image to be processed. Here, the lung image to be processed may be layer scan data obtained from an imaging device, such as a CT scanner. The lung image to be processed may be a CT image. At the same time, the lung image to be processed may also be an MRI lung image, a CT-PET lung image, etc. Those skilled in the art can select a suitable lung image according to needs.

[0064] In the present disclosure, before obtaining the lung region image and airway image corresponding to the lung image to be processed, it includes: obtaining the lung image to be processed; respectively performing lung region segmentation and airway extraction on the lung image to be processed to obtain a lung region image and an airway image.

[0065] In the embodiments of the present disclosure and other possible embodiments, for airway extraction (airway segmentation) of the lung image to be processed, the "Deep Airway Segmentation" module of Mimics software (Materialise, Belgium) can be used to semi-automatically extract a three-dimensional airway tree from the lung image to be processed to obtain an airway image.

[0066] In the embodiments of the present disclosure and other possible embodiments, for lung region segmentation of the lung image to be processed, the nnU-Net model can be used to segment the lung region of the lung image to be processed. Isensee et al. proposed the nnU-Net model for biomedical image segmentation, which provided the best performance in the segmentation task of the BraTS2020 challenge. This model can be automatically configured, including preprocessing, network structure, training, and postprocessing. The present disclosure pre-trained the nnU-Net model on the dataset of the LOLA11 challenge and fine-tuned it on the dataset of the present disclosure to complete the lung region segmentation task and obtain a lung region image. For those skilled in the art, fine-tuning a pre-trained model to complete the lung region segmentation task is a commonly used technical means in the art and there are no technical obstacles. Therefore, it will not be described in detail in the embodiments of the present disclosure.

[0067] Step S102: Respectively based on the preset classification model, use the lung image to be processed, the lung region image, and the airway image to obtain corresponding first classification result, second classification result, and third classification result.

[0068] In the embodiments of the present disclosure and other possible embodiments, the preset classification model may be a classification model based on deep learning. For example, one or several of classification models such as VGG16, VGG19, InceptionV3, Xception, MobileNet, AlexNet, LeNet, ZF_Net, ResNet18, ResNet34, ResNet50, ResNet_101, ResNet_152, etc.

[0069] For example, the to-be-processed lung image may be input into the trained VGG16 to obtain a corresponding first classification result, the lung region image may be input into the trained ResNet18 to obtain a corresponding second classification result, and the airway image may be input into the trained AlexNet to obtain a corresponding third classification result.

[0070] Meanwhile, the present disclosure proposes an image-based classification method, which can fully fuse the features extracted from the image and the corresponding weights to obtain excellent classification features. Specifically, in the present disclosure, the method for obtaining the corresponding first classification result, second classification result, and third classification result by using the to-be-processed lung image, the lung region image, and the airway image respectively based on the preset classification model includes: based on the preset classification model, respectively performing feature extraction on the to-be-processed lung image, the lung region image, and the airway image to obtain corresponding first group of features, second group of features, and third group of features; respectively calculating weight matrices corresponding to the first group of features, second group of features, and third group of features; respectively fusing the first group of features, second group of features, and third group of features and the corresponding weight matrices to obtain first classification features, second classification features, and third classification features; respectively obtaining corresponding first classification result, second classification result, and third classification result based on the first classification features, the second classification features, and the third classification features. Among them, the first classification result, second classification result, and third classification result are whether the patient has chronic obstructive pulmonary disease or not.

[0071] For example, the preset classification model selects VGG16. Based on VGG16, feature extraction is respectively performed on the lung image to be processed, the lung region image, and the airway image to obtain corresponding first group of features, second group of features, and third group of features. The first group of features, second group of features, and third group of features are respectively input into a fully connected layer (FC) to obtain a first weight matrix, a second weight matrix, and a third weight matrix corresponding to the first group of features, second group of features, and third group of features. The first group of features, second group of features, and third group of features are respectively multiplied by the corresponding first weight matrix, second weight matrix, and third weight matrix to obtain a first classification feature, a second classification feature, and a third classification feature. Further, based on the first classification feature, the second classification feature, and the third classification feature respectively, corresponding first classification result, second classification result, and third classification result are obtained.

[0072] In the embodiments of the present disclosure and other possible embodiments, the classification model for generating the first classification result, second classification result, and third classification result may be a classification model based on machine learning, such as one or several of support vector machine (SVM), multi-layer perceptron (MLP), random forest (RF), K-nearest neighbor (KNN), logistic regression (LR), decision tree (DT), gradient boosting (GB), linear discriminant analysis (LDA), etc.; it may be a classification model of deep learning.

[0073] For example, the first classification feature, the second classification feature, and the third classification feature are respectively input into a multi-layer perceptron (MLP) to obtain a first classification result, a second classification result, and a third classification result corresponding to the first classification feature, the second classification feature, and the third classification feature.

[0074] In the present disclosure, the method of respectively fusing the first group of features, second group of features, and third group of features and the corresponding weight matrices to obtain the first classification feature, second classification feature, and third classification feature includes: respectively performing matrix multiplication operations on the first group of features, second group of features, and third group of features and the corresponding weight matrices to obtain the first classification feature, second classification feature, and third classification feature.

[0075] For example, the dimensions of the first group of features, second group of features, and third group of features are k1×N1, k2×N2, and k3×N3 respectively, and the dimensions of the first group of features, second group of features, and third group of features and the corresponding first weight matrix, second weight matrix, and third weight matrix are k1×1, k2×1, and k3×1 respectively. Matrix multiplication operations are respectively performed on the first group of features, second group of features, and third group of features and the corresponding first weight matrix, second weight matrix, and third weight matrix to obtain the dimensions of the first classification feature, second classification feature, and third classification feature as 1×N1, 1×N2, and 1×N3.

[0076] Figure 2 Shows a schematic diagram of the network structure corresponding to the preset classification model according to an embodiment of the present disclosure. As Figure 2 shown, the proposed preset classification model is essentially a model corresponding to a multi-instance (MIL) method of an attention mechanism, Figure 2 only shows the first classification result corresponding to the lung image to be processed. By replacing the lung image to be processed with the lung region image and the airway image, the second classification result and the third classification result corresponding to the lung region image and the airway image can be obtained.

[0077] In the embodiments of the present disclosure and other possible embodiments, some features of multi-instance learning (MIL) are applicable to medical applications. As a weakly supervised learning method, MIL generally includes instance-level and individual-level methods. In the instance-level method, all instances are considered to contribute equally to the prediction of the individual label, and the prediction of the instance is carried out through aggregation and voting for individual prediction, while the individual method is designed to directly classify the individual. The individual method can reduce the annotation workload because it is not necessary to perform pixel and instance annotation.

[0078] In the embodiments of the present disclosure and other possible embodiments, an attention-based MIL method proposed by the present disclosure, for example, the lung image to be processed such as a CT image, combines the intensity of the CT image with the morphology of the airway and the lung region to identify chronic obstructive pulmonary disease. The main contributions of this study are as follows. First, an MIL model with an attention mechanism is constructed to classify the subjects (individuals), and the attention mechanism is used to weight the selected slices (instances) for each subject. Second, multi-view snapshots of the three-dimensional airway tree and the lung region are used as morphological information to improve the recognition performance of chronic obstructive pulmonary disease. Finally, a logistic regression (LR) model is used to integrate the pre-classification of the lung image to be processed, the lung region image, and the airway image to generate the final output. As a weakly supervised learning method, the attention mechanism-guided MIL method has the potential to provide an effective tool for the early detection of chronic obstructive pulmonary disease.

[0079] In Figure 2 it, using the feature extraction module, the instances of the lung image to be processed are transformed into k-dimensional embedding vectors (first classification features) Then, using the multi-layer perceptron u T tanh(WH T ) and the softmax layer generate an attention weight matrix α from the embedding vector H with k dimensions. Finally, by applying a function f(·) to the aggregated k instance-level feature vectors, a joint bag layer representation (first classification feature) z′ is generated, and the equation is defined as follows.

[0080] α = Softmax[u Ttanh(WH T ) (1)

[0081]

[0082] wherein includes k instance feature vectors, and are the learning parameters of the MIL module, and h is the dimension of the hidden layer.

[0083] Finally, the fully connected layer FC is the output layer, which is divided into two categories. The cross-entropy loss is used as the loss function in the model.

[0084]

[0085] wherein, y i represents the label of sample i, and p i represents the probability of predicting a positive output (having chronic obstructive pulmonary disease).

[0086] Meanwhile, in the embodiments of the present disclosure and other possible embodiments, the lung region image or the airway image can also obtain the second classification feature and the third classification feature through the above method or model, and details are not described herein again.

[0087] In the embodiments of the present disclosure and other possible embodiments, the method for respectively extracting features from the to-be-processed lung image, the lung region image, and the airway image to obtain the corresponding first group of features, second group of features, and third group of features includes: obtaining a set feature extraction model; based on the set feature extraction model, respectively extracting features from the to-be-processed lung image, the lung region image, and the airway image to obtain the corresponding first group of features, second group of features, and third group of features.

[0088] In the embodiments of the present disclosure and other possible embodiments, the set feature extraction model can be a feature extraction model based on deep learning. For example, classification models such as VGG16, VGG19, InceptionV3, Xception, MobileNet, AlexNet, LeNet, ZF_Net, ResNet18, ResNet34, ResNet50, ResNet_101, ResNet_152, etc. are one or several. Specifically, before classification, the corresponding features are saved as the corresponding first group of features, second group of features, and third group of features.

[0089] For example, the to-be-processed lung image can be input into a pre-trained VGG16 (with the classification layer removed) to obtain a corresponding first set of features, the lung region image can be input into a pre-trained ResNet18 (with the classification layer removed) to obtain a corresponding second set of features, and the airway image can be input into a pre-trained AlexNet (with the classification layer removed) to obtain a corresponding third set of features.

[0090] For another example, the to-be-processed lung image, the lung region image, and the airway image can be respectively input into a pre-trained VGG16 (with the classification layer removed) to obtain a corresponding first set of features, second set of features, and third set of features.

[0091] In the embodiments of the present disclosure and other possible embodiments, the method for respectively extracting features from the to-be-processed lung image, the lung region image, and the airway image to obtain corresponding first, second, and third sets of features includes: obtaining a preset feature extraction model, training the preset feature extraction model to obtain a trained feature extraction model; and based on the trained feature extraction model, respectively extracting features from the to-be-processed lung image, the lung region image, and the airway image to obtain corresponding first, second, and third sets of features.

[0092] In the embodiments of the present disclosure and other possible embodiments, the above CNN model requires a large amount of labeled data to train its weights and biases. However, in medical image tasks, it is difficult to meet this condition. Transfer learning has been proven to be a better method than training from scratch. Here, the modified VGG-16 (with the classification layer removed) feature extraction module was pre-trained on the ImageNet dataset (1.2M training data) and fine-tuned on our dataset. The present disclosure also adopted the same transfer learning strategy for all other comparison networks as that of VGG-16.

[0093] For example, the feature extraction part uses a pre-trained VGG-16 model. In the embodiments of the present disclosure, only the convolutional layer part of VGG-16 is retained, and the last three fully connected (FC) layers are replaced with an attention MIL pooling module in our model to obtain the structure of the modified retained VGG-16. It includes 13 convolutional layers and 5 max pooling layers. The 13 convolutional layers form four convolutional blocks. The conv1 block has two convolutional layers in sequence, and the size of the resulting feature map is the same as that of the input image, with a dimension of 64, and the number of convolutional kernels used in each convolutional layer is also 64. The conv2 block also has two convolutional layers arranged in sequence, and its output size is 128. The conv3, conv4, and conv5 blocks each have three convolutional layers in sequence, with 256, 512, and 512 convolutional kernels respectively. Max pooling is performed on each conv block, which reduces the size of the feature map by half.

[0094] Table 1 shows the detailed parameter information of the structure of the retained VGG-16 after modification.

[0095]

[0096] In the present disclosure, before respectively performing feature extraction on the to-be-processed lung image, the lung region image, and the airway image to obtain corresponding first group of features, second group of features, and third group of features, respectively determining a first image corresponding to the to-be-processed lung image, and / or the lung region image, and / or the airway image for which feature extraction is to be performed, and / or a second image, and / or a third image, the determination method includes: deleting non-lung images in the to-be-processed lung image to obtain an image containing the lung; extracting the image containing the lung according to the obtained set number to obtain a first image corresponding to the to-be-feature-extracted; performing three-dimensional reconstruction on the lung region image to obtain a three-dimensional lung region image; taking pictures of the three-dimensional lung region image at a plurality of first set angles to obtain a second image corresponding to a plurality of first two-dimensional snapshots; performing three-dimensional reconstruction on the airway image to obtain a three-dimensional airway image; taking pictures of the three-dimensional airway image at a plurality of second set angles to obtain a third image corresponding to a plurality of second two-dimensional snapshots. Among them, the first set angle may be one or several of a front view, a back view, and a 45-degree oblique view of the three-dimensional airway (tree) image or other possible views; the second set angle may be one or several of front, back, left, right, up, and down views or views at other angles.

[0097] In the embodiments of the present disclosure and other possible embodiments, the method for extracting the image containing the lung according to the obtained set number to obtain a first image corresponding to the to-be-feature-extracted includes: longitudinally dividing the image containing the lung into the set number of sub-parts on average; randomly selecting (extracting) one slice from each sub-part to obtain a first image corresponding to the to-be-feature-extracted.

[0098] For example, the to-be-processed lung image may be the to-be-processed lung image, with the extra-pulmonary part of the CT image sequence deleted, and only the part containing the lung region retained. Then, the preprocessed CT image is further longitudinally divided into k (set number) sub-parts on average. Randomly select one CT slice from each sub-part and define it as an instance. These k instances form a new individual and are used as the input of the MIL model (multi-instance model with attention mechanism). Then, the k instance images are scaled to 224×224 pixels and converted to the.npy format using Python (version 3.9), SimpleITK (version 2.1.1), and NumPy (version 1.22.1). Each.npy file corresponds to an individual.

[0099] Figure 3 Schematic diagram showing a second image and a third image corresponding to a lung region (lung field) image and an airway (airway tree) image according to an embodiment of the present disclosure. As Figure 3 shown, a snapshot is intercepted on the basis of the above segmentation. Snapshots of the front view, back view, and 45-degree oblique view (i.e., F, B, and I views) of the three-dimensional airway tree are obtained. In addition, two-dimensional snapshots in the front, back, left, right, top, and bottom views (i.e., A, P, L, R, S, and D views, respectively) of the lung region are also obtained. Using the three-dimensional display module of Slicer (https: / / www.slicer.org / ), multi-view snapshots are generated using a Python script program. Automatic batch processing is performed on all COPD patients and HC subjects. The two-dimensional snapshots used in the present disclosure are grayscale images, and their size is set to 224×224 in all snapshots. Among them, (a)-(c) are the front view, back view, and 45-degree oblique view of the three-dimensional airway tree, respectively, and (d)-(i) are the front, back, left, right, top, and bottom views of the lung region, respectively.

[0100] Step S103: Based on the first classification result, the second classification result, and the third classification result, identify COPD.

[0101] Figure 4 Schematic diagram showing a specific implementation of the processing method of the lung image according to an embodiment of the present disclosure. In the present disclosure Figure 4 the method of identifying COPD based on the first classification result, the second classification result, and the third classification result includes: respectively determining first probability values, second probability values, and third probability values corresponding to the first classification result, the second classification result, and the third classification result; performing regression analysis on the first probability value, the second probability value, and the third probability value to identify COPD.

[0102] For example, input the first probability value, the second probability value, and the third probability value into a logistic regression model, and use the logistic regression model to identify COPD. Among them, logistic regression (LR) can be regarded as an extension of linear regression, which reflects the relationship between multiple independent variables X and the result Y, and is restricted within the interval [0,1] through the sigmoid function.

[0103] In the present disclosure, statistics are performed on the first classification result, the second classification result, and the third classification result to obtain a first value corresponding to being determined as COPD and a second value corresponding to being determined as non-COPD; if the first value is greater than the second value, it is determined as COPD; otherwise, it is determined as non-COPD.

[0104] For example, if the first value corresponding to being determined as COPD is 2 and the second value corresponding to being determined as non-COPD is 1, then it is determined as COPD.

[0105] The execution subject of the method for processing lung images can be a lung image processing device. For example, the method for processing lung images can be executed by a terminal device, a server, or other processing devices. Among them, the terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the method for processing lung images can be implemented by a processor calling computer-readable instructions stored in a memory.

[0106] Those skilled in the art can understand that in the above method of the specific implementation manner, the writing order of each step does not mean a strict execution order and does not constitute any limitation to the implementation process. The specific execution order of each step should be determined according to its function and possible internal logic.

[0107] Figure 5 The block diagram of the lung image processing device according to an embodiment of the present disclosure is shown, as Figure 5 shown, the lung image processing device includes: an acquisition unit 101, configured to acquire a preset classification model, a lung image to be processed, and its corresponding lung region image and airway image; a classification unit 102, configured to respectively obtain corresponding first classification results, second classification results, and third classification results based on the preset classification model by using the lung image to be processed, the lung region image, and the airway image; and an identification unit 103, configured to perform the identification of chronic obstructive pulmonary disease based on the first classification result, the second classification result, and the third classification result.

[0108] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method for processing lung images described in the above method embodiments. Its specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0109] The embodiments of the present disclosure also propose a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented. The computer-readable storage medium can be a non-volatile computer-readable storage medium.

[0110] The embodiments of the present disclosure also propose an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to execute the above method for processing lung images. The electronic device can be provided as a terminal, a server, or other forms of devices.

[0111] The classification performance of the embodiments of the present disclosure is evaluated by accuracy (ACC), sensitivity (SEN), specificity (SPE), positive predictive value (PPV), negative predictive value (NPV), receiver operating characteristic (ROC) curve, and area under the curve (AUC) (including 95% confidence interval (CI)).

[0112] The embodiments of the present disclosure conducted four comparative experiments to study the performance of different numbers of instances for each individual, different feature extraction networks, adding airway and lung region snapshots as morphological information, and different fusion methods of the slice-CT model (a preset classification model corresponding to the lung image to be processed), the snapshot-airway model (a preset classification model corresponding to the airway image), and the snapshot-lung-field model (a preset classification model corresponding to the lung region image).

[0113] First, to determine the optimal number of instances for each individual, different numbers of instances of 10, 15, 20, and 25 in Dataset 1 were studied. These different settings enabled us to understand the impact of the number of instances on the performance of the attention-guided MIL model.

[0114] Second, we adopted four commonly used networks, such as AlexNet, VGG, ResNet18, ResNet26, and Mobilenet_v2, as the networks for feature extraction and compared them with VGG-16. All the networks were pre-trained on the ImageNet dataset (1.2M training data).

[0115] Third, we studied the contribution of airway and lung region information to the proposed method. Previous studies have shown that multi-view snapshots of the airway are an effective method for identifying chronic obstructive pulmonary disease (COPD). Therefore, three experiments were conducted here: only airway snapshots, only lung region snapshots, and a combination of CT images with airway and lung region snapshots.

[0116] Fourth, in terms of the fusion method, the LR method and the majority voting method were adopted. The majority voting method is a convenient, fast, and efficient method that integrates the predictions of multiple MIL models and obtains the final classification result. Here, each model of the MIL model was trained separately, and the final prediction was obtained by taking the majority of the single-output predictions. Specifically, in the slice-CT, snapshot-airway, and snapshot-lung-field models, if two models predict the label as "COPD", the final voting prediction result will be "COPD"; otherwise, it will be "HC".

[0117] The ten-fold cross-validation method is adopted to evaluate the proposed model. In each fold, 80%, 10%, and 10% of the individuals are used for training, validation, and testing, respectively. Due to the cross-validation strategy, all cases in dataset 1 are test cases, and dataset 2 (201 subjects, including 101 with chronic obstructive pulmonary disease and 100 healthy controls) is used as an external independent dataset for testing. The batch size is 16, the initial learning rate is 0.001, and the momentum parameter is 0.9. The datasets in training are augmented by flipping, shearing, and random rotation in both horizontal and vertical directions. Additionally, to further alleviate overfitting, the early stopping method is adopted when the ACC does not increase in five consecutive iterations.

[0118] Table 2 Performance of Different Numbers of Instances on Dataset 1

[0119]

[0120]

[0121] As shown in Table 2, 20 instances per individual achieved better performance (ACC: 88.1%, SEN: 85.3%, SPE: 91.2%, PPV: 91.5%, NPV: 84.7%) than 10, 15, and 25 instances per individual. Therefore, the CT sequences of each subject are divided into 20 sub-parts, and one slice is randomly selected from each sub-part as an instance for each individual (one subject) in the MIL method.

[0122] Table 3 Performance of Different Preset Feature Extraction Models (Backbones) on Dataset 1

[0123]

[0124] As shown in Table 3, the feature extraction performances of five networks in the attention-guided MIL model are compared. For dataset 1, the modified VGG-16 achieved better performance with an accuracy of 88.1%, higher than Alexnet (84.2%), Resnet18 (81.7%), Resnet26 (69.2%), and Mobilenet_v2 (73.3%). In terms of SPE, PPV, and NPV, the modified VGG-16 produced comparable or higher performance than other networks (Table 4). Additionally, the sensitivity of Resnet26 reached 92.6%, higher than that of the modified VGG-16 (85.3%), Alexnet (86.8%), Resnet18 (90.0%), and Mobilenet_v2 (64.7%).

[0125] In the embodiments of the present disclosure, the ROC of all considered networks in terms of feature extraction. Our method (VGG-16) achieved higher AUCs than Alexnet (0.92±0.02, 95% CI: 0.88-0.94), Resnet18 (0.91±0.02, 95% CI: 0.88-0.94), Resnet26 (0.85±0.02, 95% CI: 0.70-0.88), and Mobilenet_v2 (0.82±0.03, 95% CI: 0.77-0.86).

[0126] In the embodiments of the present disclosure, the performance of three models and the fusion method for Dataset 1 is shown in Table 4. Among the three single MIL models, the snapshot-lung-field MIL model performed better than the other two models (ACC = 90.0%, SEN: 89.5%, SPE: 90.6%, PPV: 91.4%, NPV: 88.5%), and the slice-CT MIL model performed the worst (ACC: 88.1%, SEN: 85.3%, SPE: 91.2%, PPV: 91.5%, NPV: 84.7%).

[0127] In the embodiments of the present disclosure, when the three MIL models were fused, the performance was greatly improved. The accuracy (95.8%) obtained by the LR model was higher than that of the majority voting method (95.0%). Therefore, adding snapshots of the airway and lung regions as morphological information to CT images with intensity information is beneficial for the identification of COPD.

[0128] Table 4 Performance of three models and the fusion method for Dataset 1

[0129]

[0130] In the embodiments of the present disclosure, the proposed method was validated using an external independent Dataset 2, which came from different centers and had CT images of different qualities. For Dataset 2, VGG-16 was selected as the feature extraction method, and the LR model was used to combine the slice-CT, snapshot-airway, and snapshot-lung-field models. As shown in Table 5, the method proposed in the present disclosure achieved a high ACC of 83.1%, SEN of 77.2%, SPE of 89.0%, PPV of 87.6%, and NPV of 79.5%. In addition, 78 out of 101 COPD cases and 89 out of 100 HC cases were correctly identified, showing good generalization ability.

[0131] Table 5 Performance of the method proposed in the present disclosure for Dataset 2

[0132]

[0133] In summary, the MIL method proposed in this disclosure identifies patients with chronic obstructive pulmonary disease (COPD) by fusing CT image intensity information and multi-view snapshots of the airway and lung regions as morphological information. Using the proposed new method, an ACC of 95.8% was achieved on Dataset 1 used as internal testing and an ACC of 83.1% was achieved on Dataset 2 used as external validation. On different centers and CT devices, the MIL-guided method demonstrated excellent robustness and generalization ability.

[0134] The development of deep learning in computer vision has been increasingly applied in medical image analysis. In addition, due to the easy availability of weak labels, MIL is popular in medical applications such as the diagnosis, detection, and segmentation of lesions. Similar to our study, using the MIL method for the overall diagnosis of COPD patients may only require the labels of the subjects, which are easier to obtain than the labels of each instance (CT slice). In the experiment, limited by the GPU memory and the speed of accelerating training, k slices were randomly selected from each CT scan, and the entire CT scan was not selected. The increase in the number of instances did not improve the classification performance, probably because of the similarity shared among the instances of the same individual. Therefore, a high accuracy of 88.1% was achieved with a number of 20 instances.

[0135] The good performance of using the proposed attention-guided MIL method to identify COPD is attributed to two methodological advantages. First, the CNN module based on transfer learning is a powerful tool for feature extraction in detection, classification, and segmentation tasks. Using the proposed model pre-trained on ImageNet and fine-tuned on our dataset, the generated features obtain discriminability. Previous studies have emphasized the potential advantage of the depth of the convolutional layer for feature discrimination. The neurons in the shallow network layer learn simple features such as corners or stripes, and the middle layer learns to detect parts of objects. In addition, deeper layers learn to detect high-level semantic features (concepts). Deep CNNs with different depths and architectures have different representation capabilities. Therefore, we considered various scenarios regarding the network and transfer learning techniques with the aim of finding a suitable feature extraction module. We adopted five deep CNNs (AlexNet, VGG, ResNet18, ResNet26, and Mobilenet_v2) and made comparisons. The modified VGG-16 module helped our model achieve a relatively good accuracy of 88.1%.

[0136] Next is the attention-based pooling module. MIL models usually utilize max pooling or average pooling. The above pooling methods are non-trainable and may limit their applicability. Here, we propose to use a two-layer neural network for MIL pooling. In our network, the weighted average of instances is determined, and the sum of the weights must equal 1. This operation enables the model to represent the similarity or dissimilarity among instances in an individual.

[0137] Since previous studies have determined the feasibility of using snapshots of the airway tree extracted from CT images in COPD identification using deep CNNs, we conducted experiments using snapshots of the airway and lung regions extracted from CT images as morphological information. According to previous studies, COPD is a complex, multi-system disease that can correspond to functional impairment of the airways. Therefore, airway remodeling and destruction of the lung parenchyma are very common in COPD patients. In the study by Du et al., airway information was only utilized in the deep CNN method, and the features extracted by their CNN model were discriminative for COPD classification. Their model achieved an accuracy of 88.6%. In addition, Sun et al. established an attention-based MIL model that only used CT images for COPD detection, obtaining an AUC of 0.934. In our study, we combined the advantages of CT images and airway information in COPD identification and updated the lung regions as morphological information. The research results show that adding airway and lung region information is beneficial for identifying COPD from HC because it helps to greatly improve the accuracy.

[0138] The attention-guided MIL aggregation module proposed in this study can provide good interpretability. The attention weight of each instance indicates its importance to the final prediction. While predicting COPD or HC, the proposed attention-guided MIL model also outputs the attention weight of each instance. This attention weight represents the importance of this instance to the final prediction. The distribution of attention weights is different between the COPD group and the HC group. For the COPD group, the attention weight varies with the position of the instance. From instance 1 to 20 (from the front to the back), it is the largest at instance 1, drops to the minimum at instance 5, starts to increase and reaches the second peak at instance 16, and then drops. For the HC group, the attention weight generally remains constant along the position of the instance. The attention weight value is higher at the front position of the instance, which may be related to the predominance of the upper region of emphysema, a common feature of COPD. The attention weight of the posterior instances is lower, probably due to the lower lung regions of these instances. The evenly distributed instance values in HC subjects represent approximately equal contributions to the final bag prediction.

[0139] Figure 6It is a block diagram of an electronic device 800 shown according to an exemplary embodiment. For example, the electronic device 800 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or other terminals.

[0140] Referring to Figure 6 , the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0141] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0142] The memory 804 is configured to store various types of data to support the operation of the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0143] The power component 806 provides power to various components of the electronic device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0144] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0145] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0146] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module, and the peripheral interface module can be a keyboard, a click wheel, buttons, etc. These buttons can include but are not limited to: a home button, a volume button, a power button, and a lock button.

[0147] The sensor component 814 includes one or more sensors for providing a status assessment of various aspects of the electronic device 800. For example, the sensor component 814 can detect the on / off state of the electronic device 800, the relative positioning of components, such as the display and the keypad of the electronic device 800. The sensor component 814 can also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor component 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 can also include a light sensor, such as a CMOS or a CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0148] The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices in a wired or wireless manner. The electronic device 800 can access a communication standard-based wireless network, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0149] In an exemplary embodiment, the electronic device 800 can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0150] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, and the above computer program instructions can be executed by a processor 820 of the electronic device 800 to complete the above method.

[0151] Figure 7 is a block diagram of an electronic device 1900 shown according to an exemplary embodiment. For example, the electronic device 1900 can be provided as a server. Referring to Figure 7 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by a memory 1932 for storing instructions executable by the processing component 1922, such as application programs. The application programs stored in the memory 1932 can include one or more modules each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0152] The electronic device 1900 may further include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.

[0153] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, and the computer program instructions can be executed by a processing component 1922 of the electronic device 1900 to complete the above method.

[0154] The present disclosure may be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0155] A computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium used herein is not construed as being a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0156] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices, or downloaded to an external computer or external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0157] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, including object - oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand - alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or, alternatively, may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of the present disclosure.

[0158] Aspects of the present disclosure are described herein with reference to the flowchart and / or block diagram of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowchart and / or block diagram, and combinations of blocks in the flowchart and / or block diagram, can be implemented by computer - readable program instructions.

[0159] These computer - readable program instructions can be provided to a processor of a general - purpose computer, a special - purpose computer, or other programmable data - processing apparatus to produce a machine such that the instructions, when executed by the processor of the computer or other programmable data - processing apparatus, create a means for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer - readable program instructions can also be stored in a computer - readable storage medium, which causes a computer, a programmable data - processing apparatus, and / or other devices to operate in a particular manner, so that the computer - readable medium storing the instructions comprises a manufacture, which includes instructions for implementing various aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0160] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, causing a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0161] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of code, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0162] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the technical improvement of technologies in the market, or to enable other ordinary skilled artisans in the art to understand the embodiments disclosed herein.

Claims

1. A method for processing lung CT images, characterized in that, it includes: Obtain a preset classification model, a lung CT image to be processed, a lung region CT image corresponding to the lung CT image to be processed, and an airway CT image corresponding to the lung CT image; wherein, the lung CT image to be processed, the lung region CT image, and the airway CT image respectively represent lung image intensity information, morphological information corresponding to the lung region, and morphological information corresponding to the airway; Respectively perform feature extraction on the lung CT image to be processed, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features; respectively calculate the weight matrices corresponding to the first group of features, second group of features, and third group of features; respectively perform matrix multiplication on the first group of features, second group of features, and third group of features and the corresponding weight matrices to obtain first classification features, second classification features, and third classification features; respectively based on the preset classification model, use the first classification features, the second classification features, and the third classification features to obtain first classification results, second classification results, and third classification results corresponding to the identification of chronic obstructive pulmonary disease (COPD); Perform regression analysis based on the first probability value, second probability value, and third probability value corresponding to the first classification result, second classification result, and third classification result to identify COPD; or, perform statistics on the first classification result, second classification result, and third classification result to obtain a first value corresponding to being determined as COPD and a second value corresponding to being determined as non-COPD; if the first value is greater than the second value, then determine it as COPD; otherwise, determine it as non-COPD.

2. The method for processing lung CT images according to claim 1, characterized in that, before the step of obtaining a preset classification model, a lung CT image to be processed, a lung region CT image corresponding to the lung CT image to be processed, and an airway CT image corresponding to the lung CT image to be processed, it includes: Obtain a lung CT image to be processed; Respectively perform lung region segmentation and airway extraction on the lung CT image to be processed to obtain a lung region CT image and an airway CT image.

3. The method for processing lung CT images according to any one of claims 1 or 2, characterized in that, before the step of respectively performing feature extraction on the lung CT image to be processed, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features, determining the first CT image corresponding to the lung CT image to be processed for which feature extraction is to be performed includes: Delete non-lung CT images in the lung CT image to be processed to obtain a CT image only containing the lung; According to the obtained set number, extract the CT image only containing the lung to obtain the first CT image corresponding to the feature extraction to be performed; Among them, the method for extracting the first CT image corresponding to the feature extraction to be performed from the CT image containing only the lungs according to the obtained set number includes: evenly dividing the CT image containing only the lungs into sub-parts corresponding to the set number along the longitudinal direction; randomly selecting a slice from each of the sub-parts to obtain the first CT image corresponding to the feature extraction to be performed.

4. The method for processing a lung CT image according to any one of claims 1 or 2, characterized in that before respectively performing feature extraction on the to-be-processed lung CT image, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features, determining the second CT image corresponding to the lung region CT image for which feature extraction is to be performed includes: performing three-dimensional reconstruction on the lung region CT image to obtain a three-dimensional lung region CT image; taking pictures of the three-dimensional lung region CT image at a plurality of first set angles to obtain second CT images corresponding to a plurality of first two-dimensional snapshots.

5. The method for processing a lung CT image according to claim 3, characterized in that before respectively performing feature extraction on the to-be-processed lung CT image, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features, determining the second CT image corresponding to the lung region CT image for which feature extraction is to be performed includes: performing three-dimensional reconstruction on the lung region CT image to obtain a three-dimensional lung region CT image; taking pictures of the three-dimensional lung region CT image at a plurality of first set angles to obtain second CT images corresponding to a plurality of first two-dimensional snapshots.

6. The method for processing a lung CT image according to any one of claims 1 or 2 or 5, characterized in that before respectively performing feature extraction on the to-be-processed lung CT image, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features, determining the third CT image corresponding to the airway CT image for which feature extraction is to be performed includes: performing three-dimensional reconstruction on the airway CT image to obtain a three-dimensional airway CT image; taking pictures of the three-dimensional airway CT image at a plurality of second set angles to obtain third CT images corresponding to a plurality of second two-dimensional snapshots.

7. The method for processing a lung CT image according to claim 3, characterized in that before respectively performing feature extraction on the to-be-processed lung CT image, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features, determining the third CT image corresponding to the airway CT image for which feature extraction is to be performed includes: performing three-dimensional reconstruction on the airway CT image to obtain a three-dimensional airway CT image; taking pictures of the three-dimensional airway CT image at a plurality of second set angles to obtain third CT images corresponding to a plurality of second two-dimensional snapshots.

8. The method for processing a lung CT image according to claim 4, characterized in that Before respectively performing feature extraction on the lung CT image to be processed, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features, determining the third CT image corresponding to the airway CT image for which feature extraction is to be performed includes: performing three-dimensional reconstruction on the airway CT image to obtain a three-dimensional airway CT image; Taking pictures of the three-dimensional airway CT image at a plurality of second set angles to obtain the third CT image corresponding to a plurality of second two-dimensional snapshots.

9. The method for processing a lung CT image according to any one of claims 1, 2, 5, 7, or 8, wherein, the method for respectively performing feature extraction on the lung CT image to be processed, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features includes: obtaining a preset feature extraction model, training the preset feature extraction model to obtain a trained feature extraction model; Based on the trained feature extraction model, respectively performing feature extraction on the lung CT image to be processed, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features.

10. The method for processing a lung CT image according to claim 3, wherein, the method for respectively performing feature extraction on the lung CT image to be processed, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features includes: obtaining a preset feature extraction model, training the preset feature extraction model to obtain a trained feature extraction model; Based on the trained feature extraction model, respectively performing feature extraction on the lung CT image to be processed, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features.

11. The method for processing a lung CT image according to claim 4, wherein, the method for respectively performing feature extraction on the lung CT image to be processed, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features includes: obtaining a preset feature extraction model, training the preset feature extraction model to obtain a trained feature extraction model; Based on the trained feature extraction model, respectively performing feature extraction on the lung CT image to be processed, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features.

12. The method for processing a lung CT image according to claim 6, wherein, the method for respectively performing feature extraction on the lung CT image to be processed, the lung region CT image, and the airway CT image to obtain corresponding first group of features, second group of features, and third group of features includes: obtaining a preset feature extraction model, training the preset feature extraction model to obtain a trained feature extraction model; Based on the trained feature extraction model, perform feature extraction on the to-be-processed lung CT image, the lung region CT image, and the airway CT image respectively, to obtain corresponding first group of features, second group of features, and third group of features.

13. A processing device for lung CT images, characterized in that, it includes: An acquisition unit, configured to acquire a preset classification model, a to-be-processed lung CT image, a lung region CT image corresponding to the to-be-processed lung CT image, and an airway CT image corresponding to the to-be-processed lung CT image; wherein, the to-be-processed lung CT image, the lung region CT image, and the airway CT image respectively represent lung image intensity information, morphological information corresponding to the lung region, and morphological information corresponding to the airway; A classification unit, configured to perform feature extraction on the to-be-processed lung CT image, the lung region CT image, and the airway CT image respectively, to obtain corresponding first group of features, second group of features, and third group of features; calculate corresponding weight matrices for the first group of features, second group of features, and third group of features respectively; perform matrix multiplication on the first group of features, second group of features, and third group of features and the corresponding weight matrices respectively, to obtain first classification features, second classification features, and third classification features; respectively based on the preset classification model, use the first classification features, the second classification features, and the third classification features to obtain first classification results, second classification results, and third classification results corresponding to the identification of chronic obstructive pulmonary disease (COPD); An identification unit, configured to perform regression analysis based on the first probability value, second probability value, and third probability value corresponding to the first classification result, second classification result, and third classification result to identify COPD; or, perform statistics on the first classification result, second classification result, and third classification result to obtain a first value corresponding to being determined as COPD and a second value corresponding to being determined as non-COPD; if the first value is greater than the second value, then determine it as COPD; otherwise, determine it as non-COPD.

14. An electronic device, characterized in that, it includes: A processor; A memory for storing instructions executable by the processor; wherein, the processor is configured to call the instructions stored in the memory to execute the method for processing lung CT images according to any one of claims 1 to 12.

15. A computer-readable storage medium, on which computer program instructions are stored, characterized in that, when the computer program instructions are executed by a processor, the method for processing lung CT images according to any one of claims 1 to 12 is implemented.

Images