Method and apparatus for analyzing dyspnea, electronic device, and storage medium

Abstract

The present disclosure relates to a method and device for analyzing dyspnea, an electronic device and a storage medium. The method for analyzing dyspnea comprises: obtaining lung function data corresponding to a patient to be analyzed and / or image features corresponding to a lung image; selecting corresponding set local features from the lung function data and / or the image features, respectively; determining global features according to the lung function data and / or the image features, respectively; and fusing the set local features and the global features to obtain fused features. Based on a set classifier, the fused features are used to complete the identification or identification and / or grading of dyspnea of the patient. The present disclosure can realize the identification or identification and / or grading of dyspnea.

Description

Technical Field

[0001] This disclosure relates to the technical field of dyspnea analysis, and more particularly to a method and apparatus for analyzing dyspnea, an electronic device, and a storage medium. Background Technology

[0002] Chronic obstructive pulmonary disease (COPD) is a common non-communicable lung disease characterized by persistent airflow limitation. Due to this characteristic, COPD patients often experience dyspnea as a complication. However, as a major symptom of COPD, dyspnea in COPD patients requires special consideration in the treatment of this vulnerable population for preclinical health management.

[0003] Pulmonary function testing (PFT) and computed tomography (CT) have become indispensable tools for assessing and diagnosing COPD. PFT and CT each have their own advantages in diagnosing and assessing COPD and complement each other. Compared with CT, PFT is a non-invasive method that can diagnose COPD in stages 0 to IV. According to the Global Initiative for Chronic Obstructive Lung Disease (GOLD) criteria accepted by the American Thoracic Society and the European Respiratory Society, the forced expiratory volume in one second (FEV1 / FVC) and FEV1% predicted by PFT are the gold standard for diagnosing COPD stages 0-IV. At the same time, in COPD patients, the objective information generated by the assessment of forced inspiration, especially FTV1, is closely related to the subjective dyspnea score after inhalation of bronchodilators [8]. In addition, compared with PFT, CT images can reflect changes in lung tissue in COPD patients. Therefore, CT is considered the most effective way to characterize and quantify COPD. Specifically, chest CT images can show that the patient has mild centrilobular emphysema and show reduced exercise tolerance in smokers without airflow limitation in PFT results. Furthermore, chest CT images can quantitatively analyze bronchial, airway, emphysema, and vascular problems in COPD patients by measuring parameters of the bronchial and vascular systems or using analytical methods for airway disease and emphysema. Based on this, chest CT images should provide more information for the diagnosis and assessment of COPD. Therefore, in 2007, radiomics was proposed to use advanced feature analysis to mine more information from medical images in order to extract more information from medical images. The value of pulmonary radiographic features in the assessment of chronic obstructive pulmonary disease (COPD) has also been confirmed. Radiomic features have potential applications in COPD, especially in the diagnosis, treatment, and follow-up of COPD, and the future direction of radiomic features in COPD is also under discussion. Currently, pulmonary radiographic features are widely used in COPD staging, COPD survival prediction, COPD presence prediction, COPD exacerbation, early COPD assessment, and COPD and resting heart rate analysis. However, the extraction of radiomic features from medical images through specific calculation formulas, preset image types, and preset categories limits the form of radiomic features. Convolutional Neural Networks (CNNs) for image classification have also developed rapidly, but the sheer volume of medical images limits the development of disease classification (binary or multi-class classification). CNN features extracted from medical images can compensate for the limitations of radiological features. Therefore, attention needs to be paid to deep CNN features to improve classification performance for accurate and / or personalized medicine. Summary of the Invention

[0004] This disclosure presents a method and apparatus for analyzing dyspnea, as well as an electronic device and storage medium.

[0005] According to one aspect of this disclosure, an analytical method for dyspnea is provided, comprising:

[0006] Acquire lung function data and / or imaging features corresponding to lung images of the patient to be analyzed;

[0007] Select corresponding local features from the lung function data and / or the image features respectively, determine global features based on the lung function data and / or the image features respectively, and fuse the local features and global features to obtain fused features;

[0008] Based on the set classifier, the fusion features are used to complete the identification or classification of the patient's breathing difficulties and / or grading.

[0009] Preferably, before acquiring the pulmonary function data and / or the imaging features corresponding to the lung images of the patient to be analyzed, the imaging features corresponding to the pulmonary function data and / or the lung images are determined, and the method for determining the imaging features includes:

[0010] The lung function of the patient was tested using a pulmonary function testing device to obtain the lung function data.

[0011] And / or,

[0012] Using imaging equipment, a chest image of the patient is obtained to obtain a lung image; the lung image is segmented to obtain a lung region image; and features are extracted from the lung region image to obtain the image features.

[0013] Preferably, the method for segmenting the lung image to obtain a lung region image includes:

[0014] Obtain a pre-defined lung region segmentation model;

[0015] Using the preset lung region segmentation model, the lung image is segmented to obtain a lung region image;

[0016] And / or,

[0017] The method for extracting features from the lung region image to obtain the image features includes:

[0018] Obtain transfer convolutional neural networks and / or radiomics computational models;

[0019] The lung region image is used to extract features using the transferred convolutional neural network and / or radiomics computational model to obtain the image features.

[0020] Preferably, the method for determining the predetermined local feature before selecting the corresponding predetermined local feature from the lung function data and / or the image features includes:

[0021] Obtain a predefined feature selection model; based on the feature selection model, select corresponding predefined local features from the lung function data and / or the image features;

[0022] or,

[0023] Obtain the set feature selection rules; based on the feature selection rules, select the corresponding set local features from multiple lung function data and / or multiple imaging features;

[0024] And / or,

[0025] The method for determining global features based on the lung function data and / or the imaging features includes:

[0026] Obtain the set feature fusion model; based on the feature fusion model, perform global fusion on the lung function data and / or the image features to obtain the corresponding global features;

[0027] And / or,

[0028] The method for fusing the set local features and global features to obtain the fused feature includes: concatenating the set local features and global features to obtain the fused feature;

[0029] And / or,

[0030] The analysis method is characterized in that it further includes: identifying or classifying chronic obstructive pulmonary disease in the patient before acquiring the lung function data and / or the imaging features corresponding to the lung images of the patient to be analyzed;

[0031] If the patient has the chronic obstructive pulmonary disease (COPD) or the patient's COPD meets the set identification and / or classification criteria, then the patient will undergo dyspnea analysis.

[0032] And / or,

[0033] A method for globally fusing the lung function data and / or the imaging features to obtain corresponding global features includes:

[0034] Get the set contribution rate;

[0035] A feature matrix is ​​constructed based on the lung function data and / or the imaging features, and multiple feature values ​​corresponding to the feature matrix are calculated.

[0036] Normalize the multiple feature values;

[0037] The normalized feature values ​​are sorted, and the cumulative contribution of the sorted and normalized feature values ​​is calculated.

[0038] When the cumulative contribution is greater than or equal to the set contribution rate, the feature vector of the feature value corresponding to the cumulative contribution is determined.

[0039] A transformation matrix is ​​constructed based on the feature vectors; the corresponding global features are obtained based on the feature matrix and the transformation matrix.

[0040] Preferably, the method for identifying or classifying chronic obstructive pulmonary disease in the patient includes:

[0041] Determine whether the patient has the corresponding lung function data;

[0042] If applicable, the patient may be identified or classified for chronic obstructive pulmonary disease based on the lung function data.

[0043] Otherwise, acquire lung images of the patient and use the lung images to identify or classify the patient for chronic obstructive pulmonary disease.

[0044] Preferably, the lung images include: inspiratory lung images and expiratory lung images;

[0045] Perform the registration operation of the inspiratory and expiratory lung images to obtain the displacement parameters corresponding to the transition from inhalation to exhalation;

[0046] The local features, global features, and displacement parameters are fused to obtain the fused features.

[0047] Preferably, before performing the registration operation of the inspiratory lung image and the expiratory lung image, lung region segmentation is performed on the inspiratory lung image and the expiratory lung image respectively to obtain the corresponding inspiratory lung region image and expiratory lung region image.

[0048] Perform the registration operation of the inspiratory lung region and the expiratory lung region to obtain the displacement parameters corresponding to the inspiratory to expiratory phases.

[0049] And / or,

[0050] The method for fusing the set local features, global features, and displacement parameters to obtain fused features includes: splicing the set local features, global features, and displacement parameters to obtain fused features.

[0051] According to one aspect of this disclosure, an analytical apparatus for dyspnea is provided, comprising:

[0052] The acquisition unit is used to acquire lung function data and / or imaging features corresponding to lung images of the patient to be analyzed.

[0053] The fusion unit is used to select corresponding set local features from the lung function data and / or the image features respectively, determine global features based on the lung function data and / or the image features respectively, and fuse the set local features and global features to obtain fused features;

[0054] The analysis unit is used to identify or classify the patient's breathing difficulties based on a set classifier and the fusion features.

[0055] According to one aspect of this disclosure, an electronic device is provided, comprising:

[0056] processor;

[0057] Memory used to store processor-executable instructions;

[0058] The processor is configured to execute the above-mentioned analysis method for respiratory distress.

[0059] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described method for analyzing respiratory distress.

[0060] This disclosure proposes a method for analyzing dyspnea. This method involves fusing local and global features corresponding to pulmonary function data and / or imaging characteristics to obtain a fused feature. Based on a defined classifier, the fused feature is used to identify or classify the patient's dyspnea. This addresses the current limitations of dyspnea analysis, which either cannot be performed effectively or lacks sufficient accuracy, hindering preclinical health management for patients with chronic obstructive pulmonary disease (COPD).

[0061] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0062] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0063] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0064] Figure 1 A flowchart illustrating a method for analyzing dyspnea according to an embodiment of the present disclosure is shown;

[0065] Figure 2 A schematic diagram of a convolutional network structure constructed according to an embodiment of the present disclosure is shown;

[0066] Figure 3 A schematic diagram of a graph convolutional network structure according to an embodiment of the present disclosure is shown;

[0067] Figure 4 A schematic diagram of a convolutional network structure for performing convolution operations on a three-dimensional difference matrix according to an embodiment of the present disclosure is shown.

[0068] Figure 5 A deep learning-based classification model according to an embodiment of the present disclosure is shown;

[0069] Figure 6 This is a block diagram illustrating an electronic device 800 according to an exemplary embodiment;

[0070] Figure 7 This is a block diagram illustrating an electronic device 1900 according to an exemplary embodiment. Detailed Implementation

[0071] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0072] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0073] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0074] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0075] This disclosure proposes a method for analyzing dyspnea. This method involves fusing local and global features corresponding to pulmonary function data and / or imaging characteristics to obtain a fused feature. Based on a defined classifier, the fused feature is used to identify or classify the patient's dyspnea. This addresses the current limitations of dyspnea analysis, which either cannot be performed effectively or lacks sufficient accuracy, hindering preclinical health management for patients with chronic obstructive pulmonary disease (COPD).

[0076] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further.

[0077] In addition, this disclosure also provides an analytical apparatus, electronic device, computer-readable storage medium, and program for dyspnea, all of which can be used to implement any of the dyspnea analytical methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding section of the method and will not be repeated here.

[0078] Figure 1 A flowchart illustrating an analysis method for dyspnea according to an embodiment of the present disclosure is shown, such as... Figure 1 As shown, the method for analyzing dyspnea includes: Step S101: acquiring pulmonary function data and / or image features corresponding to lung images of the patient to be analyzed; Step S102: selecting corresponding local features from the pulmonary function data and / or the image features, determining global features based on the pulmonary function data and / or the image features, and fusing the local and global features to obtain a fused feature; Step S103: using the fused feature based on a set classifier to identify or classify the patient's dyspnea. By fusing the local and global features corresponding to the pulmonary function data and / or image features to obtain a fused feature; and using the fused feature based on a set classifier to identify or classify the patient's dyspnea, this method addresses the current limitations of dyspnea analysis, which prevents preclinical health management of patients with chronic obstructive pulmonary disease.

[0079] Step S101: Obtain the lung function data and / or the imaging features corresponding to the lung images of the patient to be analyzed.

[0080] In embodiments of this disclosure, before acquiring the lung function data and / or the image features corresponding to the lung images of the patient to be analyzed, the lung function data and / or the image features corresponding to the lung images are determined. The method for determining the lung function data includes: performing a lung function test on the patient using a lung function testing instrument to obtain the lung function data.

[0081] In the embodiments of this disclosure and other possible embodiments, the pulmonary function testing device is a medical device used to measure the volume of air inhaled and exhaled by the lungs, which can perform pulmonary function tests and track lung health. The pulmonary function data may include one or more commonly used pulmonary function testing parameters such as FVC, FEV1, and FEV1 / FVC.

[0082] In embodiments of this disclosure, an imaging device is used to perform chest imaging on the patient to obtain a lung image; the lung image is segmented into lung regions to obtain a lung region image; and features are extracted from the lung region image to obtain the image features.

[0083] In the embodiments of this disclosure and other possible embodiments, the imaging device may be one or more conventional imaging devices such as CT, PET, MR, ultrasound, and DR, or a combination of the above conventional imaging devices, such as PET-CT. Simultaneously, using the above imaging device, chest imaging is performed on the patient to obtain corresponding lung images, such as chest CT lung images, chest PET lung images, chest MR lung images, chest ultrasound lung images, chest PET-CT lung images, and chest DR lung images.

[0084] In an embodiment of this disclosure, the method for segmenting the lung image to obtain a lung region image includes: obtaining a preset lung region segmentation model; and using the preset lung region segmentation model to segment the lung image to obtain a lung region image.

[0085] In the embodiments of this disclosure and other possible embodiments, the method for segmenting the lung image to obtain a lung region image includes: acquiring a preset segmentation model; and segmenting the lung image based on the preset segmentation model to obtain a lung region (lung parenchyma) image. The preset segmentation model is a pre-trained lung region (lung parenchyma) segmentation model, such as the lobe segmentation method, apparatus, and storage medium disclosed in application number 202010534722.0, to obtain the lobes of the left or right lungs. All lobes of the left lung are stitched together according to the anatomical structure of the left lung to obtain the left lung parenchyma; all lobes of the right lung are stitched together according to the anatomical structure of the right lung to obtain the right lung parenchyma. For subjects who have not undergone lobectomy, there are two lobes of the left lung and three lobes of the right lung. The two lobes of the left lung are stitched together according to the anatomical structure of the left lung to obtain the left lung parenchyma; the three lobes of the right lung are stitched together according to the anatomical structure of the right lung to obtain the right lung parenchyma. In the embodiments of this disclosure and other possible embodiments, the lung parenchyma includes the peripheral airways and pulmonary vessels. Alternatively, lung parenchyma (left and right lung parenchyma) images can be obtained directly using a lung region (lung parenchyma) segmentation model.

[0086] The embodiments of this disclosure and other possible embodiments also take into account subjects who have undergone lobectomy, wherein at least one of the following lobes has been removed: the left upper lobe, the left lower lobe, the right upper lobe, the right middle lobe, and the right lower lobe.

[0087] Based on the above, in the embodiments and other possible embodiments of this disclosure, lung segmentation may include: a left lung segmentation model, a right lung segmentation model, a left lung lobe missing segmentation model, and a right lung lobe missing segmentation model. Therefore, this disclosure proposes a technical solution for segmenting the left and right lungs separately. The left lung segmentation model, right lung segmentation model, left lung lobe missing segmentation model, and right lung lobe missing segmentation model can be lung segmentation models based on traditional segmentation algorithms or lung segmentation models based on deep learning, such as lung segmentation models based on U-Net or U-ResNet. The training methods of these models are techniques commonly used by those skilled in the art, and will not be described in detail here. However, it is worth noting that the method of segmenting the left and right lungs separately is proposed for cases of lobe missing lungs; currently, there is no method for segmenting the remaining lung after lobectomy. Therefore, the method of segmenting the left and right lungs separately is not a technique readily available to those skilled in the art and requires corresponding creative effort from those skilled in the art.

[0088] In embodiments of this disclosure and other possible embodiments, a method for segmenting the left and right lungs separately includes: acquiring a lung image to be processed; determining the position of the main bronchus (first-order trachea) in the lung image; dividing the lung image into a left lung image and a right lung image according to the position of the main bronchus; determining whether there are missing lobes in the left and right lung images respectively; if there are missing lobes, determining whether the missing lobe is in the left or right lung; if the missing lobe is in the left lung, acquiring a left lung lobe missing segmentation model and a right lung segmentation model, and using the left lung lobe missing segmentation model and the right lung segmentation model to segment the lung parenchyma of the missing left lung and the non-missing right lung respectively; if the missing lobe is in the right lung, acquiring a right lung lobe missing segmentation model and a left lung segmentation model, and using the right lung lobe missing segmentation model and the left lung segmentation model to segment the lung parenchyma of the missing right lung and the non-missing left lung respectively; and splicing the segmented left and right lung parenchyma according to anatomical structure to obtain the lung parenchyma. The main bronchus is the trachea that runs from the larynx to the hilum of the lung.

[0089] For example, if the lung image to be processed only contains the upper left lobe or the lower left lobe, then a segmentation model for the missing left lung lobe and a segmentation model for the right lung are obtained. The lung parenchyma is segmented using the missing left lung segmentation model and the right lung segmentation model, respectively. Finally, the segmented lung parenchyma of the left lung and the right lung parenchyma are spliced ​​together according to the anatomical structure to obtain the lung parenchyma mentioned above.

[0090] In embodiments of this disclosure and other possible embodiments, the method for determining the location of the main bronchus (first-order trachea) in the lung image and dividing the lung image into a left lung image and a right lung image based on the location of the main bronchus includes: acquiring an airway segmentation model; performing airway segmentation on the lung image to obtain an airway tree; determining the main bronchus in the airway tree and calculating the centerline of the main bronchus; and dividing the lung image into a left lung image and a right lung image based on the centerline. Simultaneously, the airway segmentation model can be an existing airway segmentation model, where airway segmentation only needs to identify the main bronchus, without requiring fine segmentation of the airways. For example, the airway segmentation model used in the method, apparatus, electronic device, and storage medium based on lung lobes and tracheal trees disclosed in application number 202010540322.0.

[0091] In embodiments of this disclosure, a method for extracting features from a lung region image to obtain the image features includes: acquiring a transferred convolutional neural network and / or a radiomics computational model; and using the transferred convolutional neural network and / or the radiomics computational model to extract features from the lung region image to obtain the image features. The image features may be radiomics features and / or convolutional features.

[0092] In the embodiments of this disclosure and other possible embodiments, the extraction of omics features from the lung parenchyma image (lung region image) can be achieved by a preset radiomics computation model. The preset radiomics computation model is an existing radiomics computation model, which can be obtained from the website https: / / pyradiomics.readthedocs.io / en / latest / index.html. The preset radiomics feature extraction model will not be described in detail here.

[0093] In the embodiments of this disclosure and other possible embodiments, the convolutional features of the lung parenchyma image (lung region image) can be extracted using a pre-trained feature extraction model; for example, the feature extraction model includes multi-layer convolution, which is used to extract the convolutional features of the lung parenchyma image. Alternatively, the convolutional features of the lung parenchyma image can be extracted using transfer learning; the convolutional neural network for transfer learning can be the segmentation model proposed in the paper Med3d: Transfer learning for 3d medical image analysis (Chen, S., K. Ma and Y. Zheng), however, only the encoding structure (downsampling) of the segmentation model, 3D ResNet10, 3D ResNet18, or 3D ResNet34, needs to be used to extract the convolutional features of the lung parenchyma image (lung region image).

[0094] Step S102: Select corresponding local features from the lung function data and / or the image features respectively, determine global features based on the lung function data and / or the image features respectively, and fuse the local features and global features to obtain fused features.

[0095] In the embodiments and other possible embodiments disclosed herein, since local features are already defined, the corresponding defined local features can be selected from the lung function data and / or the image features. However, global features need to be further determined based on the lung function data and / or the image features. For example, if the defined local features corresponding to the lung function data feature AZ are features A, B, and F, then features A, B, and F can be selected simply from the lung function data feature AZ. As another example, if the defined local features corresponding to the lung function data feature af are features a, c, and z, then features a, c, and z can be selected simply from the lung function data feature af.

[0096] In embodiments of this disclosure, the method for determining the set local features before selecting corresponding set local features from the lung function data and / or the image features includes: obtaining a set feature selection model; selecting corresponding set local features from multiple lung function data and / or multiple image features based on the feature selection model; or, obtaining a set feature selection rule; selecting corresponding set local features from multiple lung function data and / or multiple image features based on the feature selection rule. Wherein, the multiple lung function data and / or multiple image features used to determine the set local features are lung function data and / or image features corresponding to multiple dyspnea identifications or identifications and / or grading labels when constructing the model (training the classifier). Similarly, constructing the model (training the classifier) ​​also requires global features, where the multiple lung function data and / or multiple image features used for fusion are lung function data and / or image features corresponding to multiple dyspnea identifications or identifications and / or grading labels when constructing the model (training the classifier). Wherein, "multiple" refers to the number of training sets.

[0097] In embodiments of this disclosure, the method for determining global features based on the lung function data and / or the image features includes: acquiring a set feature fusion model; and globally fusing the lung function data and / or the image features based on the feature fusion model to obtain corresponding global features.

[0098] Similarly, when constructing a model (training a classifier), a method for determining global features based on multiple lung function data and / or multiple imaging features includes: obtaining a predefined feature fusion model; and, based on the feature fusion model, globally fusing the multiple lung function data and / or the multiple imaging features to obtain corresponding global features for training. For detailed implementation procedures, please refer to the description of the method for determining global features based on the lung function data and / or the imaging features.

[0099] In embodiments of this disclosure and other possible embodiments, the method for selecting corresponding set local features from multiple lung function data and / or multiple image features based on the feature selection model includes: obtaining dyspnea identification (0 or 1) or grade (0-4) corresponding to the multiple lung function data and / or the multiple image features; using the feature selection model, establishing the relationship between the multiple lung function data and / or the multiple image features and the dyspnea identification (0 or 1) or grade (0-4), and obtaining the set local features corresponding to the multiple lung function data and / or the multiple image features respectively.

[0100] For example, in embodiments of this disclosure and other possible embodiments, the set feature selection model can be the Lasso model. Using the Lasso model, based on the dyspnea identification or grading corresponding to the lung images (dyspnea or grading or whether dyspnea is present (0 - no dyspnea, 1 - dyspnea)), feature selection is performed on the lung function data and / or the radiomics features and / or the convolutional features to obtain the selected lung function data and / or radiomics features and / or the selected convolutional features (setting local features). Simultaneously, the dyspnea identification or grading can also select any two levels of the dyspnea grading, or any combination of several gradings. For example, dyspnea 0 (no dyspnea) can be used as the first label 0, and other stages other than dyspnea 0 can be used as the second label 1; alternatively, dyspnea 4 can be used as the second label 1, and other grades other than dyspnea 4 can be used as the first label 0. More specifically, the dyspnea grading as a label can be obtained from a survey using the Improved Medicine Research Council (MMRC) scale, with dyspnea grades ranging from 0 to 4; the Lasso model can be implemented using the standard Python package LassoCV (Python 3.6).

[0101] In another embodiment of this disclosure and other possible embodiments, the method of obtaining a set feature selection rule and selecting a corresponding set local feature from multiple lung function data and / or multiple imaging features (the omics features and / or the convolutional features) based on the feature selection rule includes: obtaining a set saliency and dyspnea identification (0 or 1) or grading (0-4) corresponding to the multiple lung function data and / or the multiple imaging features; calculating multiple saliency values ​​corresponding to the multiple lung function data and / or the multiple imaging features (the omics features and / or the convolutional features) under the dyspnea identification or grading; and determining the lung function data and / or imaging features corresponding to saliency values ​​less than or equal to the set saliency as set local features.

[0102] For example, regarding the dyspnea identification (labeled 0, 1) corresponding to the multiple lung function data and / or the multiple imaging features, the significance values ​​corresponding to the multiple lung function data and / or the multiple imaging features under the dyspnea identification (labeled 0, 1) are calculated respectively. The lung function data and / or imaging features corresponding to significance values ​​less than or equal to the set significance value are determined as set local features; wherein, the number of calculated significance values ​​is the number of lung function data and / or imaging features. As another example, regarding the dyspnea grades (labeled 0-4) corresponding to the multiple lung function data and / or the multiple imaging features, the significance values ​​corresponding to the multiple lung function data and / or the multiple imaging features under the dyspnea grades (labeled 0-4) need to be calculated respectively. The lung function data and / or imaging features corresponding to significance values ​​less than or equal to the set significance value across all grades are determined as set local features; wherein, the number of calculated significance values ​​is the number of lung function data and / or imaging features. The salience setting can be configured to 0.5, 0.1, 0.01, etc., and those skilled in the art can configure the salience setting according to actual needs.

[0103] In embodiments of this disclosure, the method for determining global features based on the lung function data and / or the image features includes: acquiring a predefined feature fusion model; and globally fusing the lung function data and / or the image features based on the feature fusion model to obtain corresponding global features. In embodiments of this disclosure and other possible embodiments, the feature fusion model may be a principal component analysis (PCA) model or a neural network.

[0104] Meanwhile, when constructing the model (training the classifier), the method for determining global features based on multiple lung function data and / or multiple image features includes: obtaining a set feature fusion model; and globally fusing the multiple lung function data and / or the multiple image features based on the feature fusion model to obtain corresponding global features for training.

[0105] In embodiments of this disclosure, the method for globally fusing the lung function data and / or the image features to obtain corresponding global features includes: obtaining a set contribution rate; constructing a feature matrix based on the lung function data and / or the image features, and calculating multiple feature values ​​corresponding to the feature matrix; normalizing the multiple feature values; sorting the normalized multiple feature values, and calculating the cumulative contribution of the sorted and normalized multiple feature values; determining the feature vector of the feature value corresponding to the cumulative contribution when the cumulative contribution is greater than or equal to the set contribution rate; constructing a transformation matrix based on the feature vector; and obtaining the corresponding global features based on the feature matrix and the transformation matrix. The set contribution rate can be configured to 95%, and those skilled in the art can configure the set contribution rate according to actual needs.

[0106] In embodiments of this disclosure and other possible embodiments, a predetermined contribution rate is obtained; a feature matrix A is constructed based on the lung function data and / or the imaging features. m×n =(a1,a,a3,…,a n ); where a1, a, a3, ..., a n For the lung function data and / or the imaging features, the singular value decomposition (SVD) algorithm can be used to calculate multiple eigenvalues ​​(λ1, λ2, λ3, ..., λ) corresponding to the feature matrix. n The process involves: normalizing the multiple feature values; sorting the normalized feature values ​​and calculating the cumulative contribution of the sorted and normalized feature values; and determining the feature vector of the feature value corresponding to the cumulative contribution when the cumulative contribution is greater than or equal to the set contribution rate. Construct a transformation matrix P based on the eigenvectors. k×n =(ξ1,ξ2,ξ3,…,ξ) k ) k×n The corresponding global features are obtained based on the feature matrix and the transformation matrix.

[0107] Specifically, the singular value decomposition (SVD) algorithm is used to calculate multiple eigenvalues ​​(λ1, λ2, λ3, ..., λ) corresponding to the feature matrix. n It can be obtained from Formula 1 and Formula 2.

[0108] A T A=(UΣV T ) T UΣV T =VΣ T U T UΣV T =VΣ T ΣV T =VΣ2 V T Formula 1

[0109]

[0110] Among them, U m×m and V n×n For an orthogonal matrix, ∑ m×n =(σ1,σ2,σ3,…,σ k ) is a diagonal matrix, σ i Let matrix A T The eigenvalues ​​corresponding to A, m and n are the dimensions of the matrix. Specifically, m is the number of patients, and n is the number of lung function data and / or the number of imaging features.

[0111] Specifically, the corresponding global feature B is obtained based on the feature matrix and the transformation matrix. m×n It can be obtained from Formula 3.

[0112]

[0113] Among them, b1, b, b3, ..., b k This corresponds to the global feature.

[0114] In another embodiment of this disclosure, the method for globally fusing the lung function data and / or the image features to obtain corresponding global features includes: constructing a neural network; training the neural network using multiple lung function data and / or multiple image features; and globally fusing the lung function data and / or the image features based on the trained neural network to obtain corresponding global features. The neural network includes at least an input layer, a hidden layer, and an output layer; the parameters corresponding to the input layer, hidden layer, and output layer of the neural network are trained using multiple lung function data and / or multiple image features to obtain a trained neural network.

[0115] More specifically, in the embodiments of this disclosure and other possible embodiments, the lung function data and / or the image features each correspond to their respective predetermined local features; similarly, the lung function data and / or the image features each correspond to their respective global features. For example, the lung function data corresponds to a first predetermined local feature, and the image features correspond to a second predetermined local feature; the lung function data corresponds to a first global feature, and the image features correspond to a second global-local feature. Further, if the image features include radiomics features and CNN (convolutional neural network) features, then the radiomics features and CNN (convolutional neural network) features each correspond to their respective predetermined local features and global features. For example, the radiomics features and convolutional features in the image features correspond to a second predetermined local feature and a third predetermined local feature, respectively; the radiomics features and convolutional features in the image features correspond to a second global feature and a third global feature, respectively.

[0116] In embodiments of this disclosure, the method for fusing the defined local features and global features to obtain fused features includes: concatenating the defined local features and global features to obtain the fused features. For example, when analyzing pulmonary function data or lung images corresponding to a patient, the model constructed is also trained from multiple pulmonary function data or lung images. In this case, only the defined local features and global features corresponding to the pulmonary function data or the defined local features and global features corresponding to the lung images are concatenated. As another example, when analyzing pulmonary function data and lung images corresponding to a patient, the model constructed is also trained from multiple pulmonary function data and lung images. In this case, the defined local features, global features, and global features corresponding to the pulmonary function data and lung images are concatenated.

[0117] In embodiments of this disclosure, the analysis method further includes: identifying or classifying chronic obstructive pulmonary disease (COPD) in the patient before acquiring the lung function data and / or imaging features corresponding to the lung images of the patient to be analyzed; if the patient has COPD or the patient's COPD reaches a set level, then performing dyspnea analysis on the patient. The set level can be configured to any level from 1 to 4.

[0118] In embodiments of this disclosure, the method for identifying or classifying chronic obstructive pulmonary disease (COPD) in the patient includes: determining whether the patient has corresponding lung function data; if so, identifying or classifying COPD in the patient based on the lung function data; otherwise, acquiring lung images of the patient and using the lung images to identify or classify COPD in the patient.

[0119] In embodiments of this disclosure and other possible embodiments, the identification or classification of the patient for chronic obstructive pulmonary disease (COPD) based on the lung function data may be performed according to the Global Initiative for Chronic Obstructive Pulmonary Disease (GOLD) criteria accepted by the American Thoracic Society and the European Respiratory Society (GRIS).

[0120] In embodiments of this disclosure and other possible embodiments, the method for acquiring lung images of the patient and using the lung images to identify or classify chronic obstructive pulmonary disease (COPD) in the patient includes: acquiring lung images to be processed and segmenting the lung images into lung regions to obtain lung parenchymal images; extracting omics features and convolutional features from the lung parenchymal images respectively; and performing feature selection on the omics features and convolutional features based on COPD identification or classification labels corresponding to the lung images respectively to obtain selected radiomics features and selected convolutional features; performing a first fusion operation on the selected radiomics features and the selected convolutional features to obtain a first fusion feature; and identifying and / or classifying COPD based on the first fusion feature and a preset classifier. This disclosure extracts omics features and convolutional features from the lung parenchyma image, respectively. Based on the COPD identification and / or grading labels corresponding to the lung image, feature selection is performed on the omics features and convolutional features to obtain selected radiomics features and selected convolutional features. A first fusion operation is performed on the selected radiomics features and selected convolutional features to obtain a first fused feature. Based on the first fused feature and a preset classifier, COPD identification and / or grading are then achieved. Compared with traditional methods, this disclosure incorporates convolutional features corresponding to the lung parenchyma image. By fusing radiomics features and convolutional features, the accuracy of COPD identification and / or grading and the classification performance of the classifier are improved.

[0121] The specific method for acquiring a lung image to be processed and performing lung region segmentation on the lung image to obtain a lung parenchyma image can be found in the detailed description above. In the embodiments of this disclosure and other possible embodiments, subjects who have undergone lobectomy are also considered, wherein at least one lobe of the left upper lobe, left lower lobe, right upper lobe, right middle lobe, and right lower lobe has been removed.

[0122] The omics features and convolutional features of the lung parenchyma image are extracted respectively; and, based on the COPD identification and / or grading labels corresponding to the lung image, feature selection is performed on the omics features and the convolutional features to obtain the selected radiomics features and the selected convolutional features.

[0123] In the embodiments of this disclosure and other possible embodiments, the extraction of omics features from the lung parenchyma image can be achieved through a preset radiomics feature extraction model. The preset radiomics feature extraction model is an existing radiomics computation model, which can be obtained from the website https: / / pyradiomics.readthedocs.io / en / latest / index.html. The preset radiomics feature extraction model will not be described in detail here.

[0124] In the embodiments of this disclosure and other possible embodiments, the convolutional features of the lung parenchyma image can be extracted using a pre-trained feature extraction model. For example, the feature extraction model includes multiple convolutional layers, which are used to extract the convolutional features of the lung parenchyma image. Alternatively, the convolutional features of the lung parenchyma image can be extracted using transfer learning. The transfer learning model can be the segmentation model proposed in the paper Med3d: Transfer learning for 3D medical image analysis (Chen, S., K. Ma and Y. Zheng), where only the encoding structure 3D ResNet10, 3D ResNet18, or 3D ResNet34 is needed to extract the convolutional features of the lung parenchyma image.

[0125] Meanwhile, in this disclosure, the method for selecting radiomics features and convolutional features based on COPD identification and / or grading labels corresponding to the lung images to obtain selected radiomics features and selected convolutional features includes: obtaining a radiomics selection model; and based on the radiomics selection model, selecting features on the radiomics features and convolutional features by establishing the relationship between the COPD identification and / or grading labels and the corresponding radiomics features and convolutional features to obtain selected radiomics features and selected convolutional features.

[0126] In embodiments of this disclosure and other possible embodiments, the feature selection model may be a Lasso model. Using the Lasso model, feature selection is performed on the omics features and the convolutional features based on the COPD identification and / or grading labels corresponding to the lung images, respectively, to obtain selected radiomics features and selected convolutional features. For example, if the number of omics features extracted from the lung parenchyma image is 1316, and feature selection is performed on the omics features and the convolutional features based on the COPD identification and / or grading labels corresponding to the lung images, the selected radiomics features are obtained.

[0127] The selected radiomics features and the selected convolutional features are subjected to a first fusion operation to obtain a first fused feature; and based on the first fused feature and a preset classifier, COPD is identified and / or classified. The classification label is from stage 0 to stage IV, where stage 0 indicates no COPD.

[0128] In this disclosure, the method for performing a first fusion operation on the selected radiomics features and the selected convolutional features to obtain a first fused feature includes: performing vector concatenation on the selected radiomics features and the selected convolutional features to obtain the first fused feature.

[0129] For example, if the selected radiomics features corresponding to a lung image to be processed are [1, 2, 4], where 1, 2, and 4 correspond to different names of radiomics features, and the selected convolutional features corresponding to this lung image to be processed are [0, 2, 1], then the selected radiomics features and the selected convolutional features are vector concatenated to obtain the first fusion feature as [1, 2, 4, 0, 2, 1] or [0, 2, 1, 1, 2, 4].

[0130] In this disclosure, before performing a first fusion operation on the selected radiomics features and the selected convolutional features to obtain a first fused feature, the method further includes: constructing a radiomics feature map based on the omics features of the lung parenchyma image; performing convolution processing on the radiomics feature map to obtain omics convolutional features; performing a second fusion operation on the omics convolutional features and the omics features of the lung parenchyma image to obtain a second fused feature; and performing a first fusion operation on the second fused feature and the selected convolutional features to obtain a first fused feature.

[0131] In this disclosure, the method of performing a first fusion operation on the second fusion feature and the selected convolutional feature to obtain a first fusion feature includes: performing vector concatenation on the second fusion feature and the selected convolutional feature to obtain the first fusion feature.

[0132] For example, if the second fusion feature is [1, 2, 4] and the selected convolutional feature is [0, 2, 1], then the second fusion feature and the selected convolutional feature are concatenated to obtain the first fusion feature [1, 2, 4, 0, 2, 1] or [0, 2, 1, 1, 2, 4].

[0133] In this disclosure, the method for constructing a radiomics feature map based on the omics features of the lung parenchyma image includes: taking one feature from the omics features as a base feature, and subtracting the remaining features from the base feature to obtain a radiomics feature vector; similarly, taking the remaining features from the omics features as base features to obtain corresponding radiomics feature vectors; and concatenating all the radiomics feature vectors to obtain a radiomics feature map.

[0134] For example, to obtain a second number N of omics features of the lung parenchyma image, a radiomics feature map of size N×(N-1) is constructed. The specific construction method includes: taking one feature from the omics features as a base feature, and subtracting the remaining features from the base feature to obtain a 1×(N-1) radiomics feature vector; similarly, taking N-1 features from the omics features as base features to obtain a (N-1)×(N-1) radiomics feature vector; and concatenating the radiomics feature vectors to obtain a radiomics feature map of size N×(N-1).

[0135] For example, to obtain a second number N of omics features of the lung parenchyma image and construct an N×N radiomics feature map, the specific construction method includes: taking one feature from the omics features as a base feature, subtracting the remaining features from the base feature to obtain a 1×N radiomics feature vector, wherein the first element in the radiomics feature vector is 0 (base feature minus itself); similarly, taking N-1 features from the omics features as base features to obtain a (N-1)×N radiomics feature vector; and concatenating the radiomics feature vectors to obtain an N×N radiomics feature map.

[0136] In this disclosure, the method for identifying and / or classifying COPD based on the first fusion feature and a preset classifier includes: obtaining a set ratio; dividing all lung images corresponding to the first fusion based on the set ratio to obtain a training set and a validation set; training the preset classifier using the training set; and validating the trained preset classifier using the validation set, thereby identifying and / or classifying COPD.

[0137] In embodiments of this disclosure and other possible embodiments, for example, the number of lung images to be processed is 1000 sets, the ratio is set to 7:3, and the first fusion corresponding to all the lung images is divided based on the set ratio to obtain 700 sets of lung images in the training set and 300 sets of lung images in the validation set; the preset classifier is trained using the training set; the preset classifier after training is validated using the validation set, thereby identifying and / or classifying COPD.

[0138] In embodiments of this disclosure and other possible embodiments, the method for determining the preset classifier includes: acquiring a plurality of classifiers to be determined; classifying the lung parenchyma image using omics features based on the plurality of classifiers to be determined, thereby obtaining a plurality of corresponding first set of classification indicators; determining the first classifier corresponding to the best classification indicator from the plurality of classifiers according to the plurality of corresponding first set of classification indicators; classifying the lung parenchyma image using convolutional features based on the plurality of classifiers to be determined, thereby obtaining a plurality of corresponding second set of classification indicators; determining the second classifier corresponding to the best classification indicator from the plurality of classifiers according to the plurality of corresponding second set of classification indicators; if the first classifier and the second classifier are the same, then the preset classifier is either the first classifier or the second classifier.

[0139] In the embodiments of this disclosure and other possible embodiments, the classifier to be determined may be one or more of Support Vector Machine (SVM), Multilayer Perceptron (MLP), Random Forest (RF), K-Nearest Neighbors (KNN), Logistic Regression (LR), Decision Tree (DT), Gradient Boosting (GB), Linear Discriminant Analysis (LDA), etc. Meanwhile, in the embodiments of this disclosure and other possible embodiments, the preset classifier is a Multilayer Perceptron (MLP).

[0140] In the embodiments of this disclosure and other possible embodiments, the selected radiomics features are local features, while the convolutional features obtained by convolving the radiomics feature maps are global features. Therefore, this disclosure proposes a method for fusing local and global features to obtain a second fused feature that includes both local and global features. A first fusion operation is then performed on the second fused feature and the selected convolutional features to obtain a first fused feature. Subsequently, based on the first fused feature and a preset classifier, COPD is identified and / or classified.

[0141] In this disclosure, the method of performing convolution processing on the image omics feature map to obtain omics convolution features, and performing a second fusion operation on the omics convolution features and the omics features of the lung parenchyma image to obtain a second fusion feature, includes: obtaining a first number corresponding to the selected image omics features; performing convolution processing on the image omics feature map to obtain the first number of omics convolution features; and performing an addition operation on the omics convolution features and the omics features of the lung parenchyma image to obtain the second fusion feature.

[0142] In the embodiments of this disclosure and other possible embodiments, the first number corresponding to the selected radiomics features is 106. Therefore, the radiomics feature map is convolved to obtain the first number of 106 omics convolutional features. Only when the number corresponding to the selected radiomics features is the same as the number of omics convolutional features can the omics convolutional features and the omics features of the lung parenchyma image be added together to obtain the second fusion feature.

[0143] For example, the omics features of the lung parenchyma image are [1, 2, 4], and the omics convolution features are [1, 1, 4]. The omics convolution features and the omics features of the lung parenchyma image are added together to obtain the second fusion feature [2, 3, 8].

[0144] In another embodiment and other possible embodiments of this disclosure, the method for acquiring lung images of the patient and using the lung images to identify or classify chronic obstructive pulmonary disease (COPD) includes: acquiring radiomics features corresponding to lung parenchymal images; selecting the radiomics features based on identification and / or classification labels corresponding to the lung parenchymal images to obtain selected radiomics features; determining the number of radiomics features; generating a radiomics feature map using the radiomics features; performing a convolution operation on the radiomics feature map to obtain the number of convolutional features; performing a fusion operation on the radiomics features and the convolutional features to obtain fused features; determining risk factor features of a graph convolutional network based on the fused features; performing a stitching operation on the risk factor features and the fused features to obtain stitched features; and using the stitched features to identify and / or classify COPD based on the graph convolutional network. Unlike existing technologies, this disclosure fuses local radiomics features (selected radiomics features) with global radiomics features (convolutional features) corresponding to the radiomics feature map to obtain fused features. Based on these fused features, risk factor features of the graph convolutional network are determined. The risk factor features and the fused features are then concatenated to obtain concatenated features. Based on the graph convolutional network, the concatenated features are used to identify and / or classify COPD. This fully leverages radiomics features to further identify and / or classify COPD, thereby advancing the clinical application of radiomics features in COPD identification and / or classification.

[0145] Obtain radiomics features corresponding to lung parenchyma images; select the radiomics features based on the identification and / or hierarchical labels corresponding to the lung parenchyma images to obtain the selected radiomics features; and determine the number of the radiomics features.

[0146] In embodiments of this disclosure, the method for segmenting the lung image to be processed to obtain a lung parenchyma image corresponding to the lung region includes: acquiring a preset lung region segmentation model; and segmenting the lung image to be processed based on the preset lung region segmentation model to obtain a lung parenchyma image. The lung parenchyma image includes the lung parenchyma corresponding to the left and right lungs. The preset lung region segmentation model can be a pre-trained U-net neural network or a ResU-Net neural network. In embodiments of this disclosure and other possible embodiments, the lung parenchyma includes peripheral airways and pulmonary vessels.

[0147] In the embodiments and other possible embodiments disclosed herein, before obtaining the radiomics features corresponding to the lung parenchyma image, the omics features of the lung parenchyma image are extracted. The extraction of the omics features of the lung parenchyma image can be achieved by a preset radiomics feature extraction model. The preset radiomics feature extraction model is an existing radiomics computation model, which can be obtained from the website https: / / pyradiomics.readthedocs.io / en / latest / index.html. The preset radiomics feature extraction model will not be described in detail here.

[0148] In the embodiments and other possible embodiments of this disclosure, in the method of selecting radiomics features based on the identification and / or hierarchical labels corresponding to lung parenchyma images to obtain selected radiomics features, the Lasso model is used to select the radiomics features based on the identification and / or hierarchical labels corresponding to lung parenchyma images to obtain selected radiomics features.

[0149] In the embodiments and other possible embodiments of this disclosure, the number of radiomics features corresponding to the lung parenchyma image is 1316; based on the identification and / or hierarchical labels corresponding to the lung parenchyma image, the radiomics features are selected, resulting in a number of 106 selected radiomics features.

[0150] The radiomics features are used to generate a radiomics feature map, and the radiomics feature map is convolved to obtain the number of convolutional features.

[0151] In this disclosure, unlike existing technologies, the method for generating an image omics feature map using the image omics features includes: determining the number M of the image omics features; arranging the image omics features according to a predetermined method to generate an M×M image omics feature map. In embodiments of this disclosure and other possible embodiments, the method of arranging the image omics features according to a predetermined method to generate an M×M image omics feature map includes: using the M feature elements of the image omics features as the image omics feature vector of the first row; shifting the image omics feature vector of the next row to the right of the image omics feature vector of the previous row, with the last feature element as the first feature element of this row; and finally, generating an M×M image omics feature map.

[0152] For example, if the radiomics feature is [1,2,3,4], and the number of radiomics features is 4, then the radiomics feature vector in the first row is [1,2,3,4], the radiomics feature vector in the second row is [4,1,2,3], the radiomics feature vector in the third row is [3,4,1,2,], and the radiomics feature vector in the fourth row is [2,3,4,1]. Finally, a 4×4 radiomics feature map is generated, and the first to fourth rows of the 4×4 radiomics feature map are [1,2,3,4], [4,1,2,3], [3,4,1,2,] and [2,3,4,1], respectively.

[0153] In the embodiments and other possible embodiments of this disclosure, the size of the radiomics feature map corresponding to the number of radiomics features 1316 is 1316×1316 using the above method.

[0154] In this disclosure, a method for performing a convolution operation on the radiomics feature map to obtain the number of convolutional features includes: obtaining a transfer convolution model; and using the transfer convolution model to perform a convolution operation on the radiomics feature map to obtain the number of convolutional features.

[0155] In the embodiments and other possible embodiments disclosed herein, the transfer convolution model can be selected from the segmentation model proposed in the article Med3d: Transfer learning for 3d medical image analysis (Chen, S., K. Ma and Y. Zheng). However, it is only necessary to use the encoding structure 3D ResNet10, 3D ResNet18 or 3DResNet34 in the segmentation model to perform convolution operations on the image omics feature map to obtain the number of convolutional features.

[0156] Figure 2 A schematic diagram illustrating the construction of a convolutional network according to an embodiment of this disclosure is shown. Figure 2 As shown, the constructed convolutional network includes: a first convolutional layer Cov1 (1,64,7,3,1), a second convolutional layer Cov2 (64,32,5,2,0), a third convolutional layer Cov3 (32,16,3,1,0), a fourth convolutional layer Cov4 (16,8,3,1,0), a fifth convolutional layer Cov5 (8,1,2,1,0), a pooling layer Pool (3,2,0), a first fully connected layer FC1 (11236,106), and a second fully connected layer FC2 (106,4).

[0157] Figure 2During training, the constructed convolutional network retains the second fully connected layer FC2(106,4); when the trained convolutional network is used to perform convolution operations on the image omics feature map to obtain the number of convolutional features, the second fully connected layer FC2(106,4) is deleted.

[0158] Taking the training of the constructed convolutional network as an example, for Figure 2 To provide a detailed explanation, the radiomics feature map is a single feature map. The radiomics feature map is convolved using a 7×7 convolution kernel with a stride of 3, and after edge padding, 64 feature maps are obtained. 64 feature maps are convolved using a 5×5 kernel with a stride of 2 and no padding, resulting in 32 feature maps. These 32 feature maps are then convolved using a 3×3 kernel with a stride of 1 and no padding, resulting in 16 feature maps. These 16 feature maps are then convolved using a 3×3 kernel with a stride of 1 and no padding, resulting in 8 feature maps. These 8 feature maps are then convolved using a 2×2 kernel with a stride of 1 and no padding, resulting in 1 feature map. This 1 feature map is then pooled using a 3×3 kernel with a stride of 2 and a stride of 1, resulting in a pooled feature vector with 11236 features. This pooled feature vector with 11236 features is then passed through a first fully connected layer (FC1) to obtain 106 feature vectors. After passing through the second fully connected layer FC2, four feature vectors are obtained. The operator can choose the loss function to use during training according to actual needs.

[0159] The radiomics features and the convolutional features are fused to obtain fused features; the risk factor features of the graph convolutional network are determined based on the fused features; the risk factor features and the fused features are spliced ​​together to obtain spliced ​​features.

[0160] The fusion operation on the radiomics features and the convolutional features to obtain fused features is the basis for determining the risk factor features of the graph convolutional network based on the fused features. Because the fused features are obtained by fusing the radiomics features and the convolutional features, a fusion of local features (selected radiomics features) and global features (unselected radiomics features, or the radiomics features before selection) is achieved, the fused features better reflect the radiomics features. Therefore, the method of fusing the radiomics features and the convolutional features to obtain fused features is not a technique commonly used by those skilled in the art, and requires corresponding creative effort from those skilled in the art.

[0161] In embodiments and other possible embodiments of this disclosure, unlike the prior art, the method of fusing the radiomics features and the convolutional features to obtain fused features includes: element-wise addition of the radiomics features and the convolutional features to obtain fused features.

[0162] For example, the radiomics feature is [1,1,1,3], the convolution feature is [0,1,0,3], and the radiomics feature and the convolution feature are added at the pixel level to obtain the fused feature [1,2,1,6].

[0163] In the embodiments and other possible embodiments of this disclosure, unlike the prior art, this disclosure proposes three methods for determining the risk factor characteristics of a graph convolutional network based on the fusion features. How to determine the risk factor characteristics of a graph convolutional network directly affects the generation of the edge constraint matrix E, and thus affects the adjacency matrix. Therefore, the above-mentioned methods for determining the risk factor characteristics of a graph convolutional network based on the fusion features are not conventional techniques used by those skilled in the art, and require corresponding creative effort from those skilled in the art.

[0164] In embodiments and other possible embodiments of this disclosure, the first method, the method for determining the risk factor features of a graph convolutional network based on the fusion features, includes: obtaining a generalized linear model (GLM); using the GLM, obtaining the R^2 value of each feature element in the fusion features based on the fusion features and corresponding identification and / or hierarchical labels; sorting the feature elements in the fusion features from largest to smallest using the R^2 values; and extracting the first predetermined number E features after sorting as the risk factor features of the graph convolutional network. The predetermined number E is generally 2-6.

[0165] In the embodiments and other possible embodiments of this disclosure, the second method, the method for determining the risk factor features of the graph convolutional network based on the fusion features, further includes: obtaining a Lasso model; using the Lasso model, obtaining screening features of the fusion features based on the fusion features and corresponding identification and / or hierarchical labels; sorting the screening features from largest to smallest using the regression coefficients of the screening features, and selecting the top E screening features after sorting as the risk factor features of the graph convolutional network. The set number E is generally 2-6, and the number of screening features is less than the number of fusion features.

[0166] In the embodiments and other possible embodiments of this disclosure, the third method, the method for determining the risk factor features of a graph convolutional network based on the fusion features, includes: obtaining an independent component analysis (PCA) model; using the PCA model, based on the fusion features and corresponding identification and / or hierarchical labels, obtaining a predetermined number E dimensionality-reduced features of the fusion features; and using the predetermined number E dimensionality-reduced features as the risk factor features of the graph convolutional network. The predetermined number E is generally 2-6, and the number of dimensionality-reduced features is less than the number of fusion features.

[0167] In the embodiments and other possible embodiments of this disclosure, the specific implementation of the method for concatenating the risk factor features and the fusion features to obtain concatenated features is as follows. If the value of E is 4, then the 4 risk factor features and 106 fusion features are concatenated to obtain a concatenated feature (vector) with a dimension of 110.

[0168] Based on the graph convolutional network, the splicing features are used to achieve the identification and / or classification of COPD.

[0169] Figure 3 A schematic diagram of a graph convolutional network structure according to an embodiment of the present disclosure is shown. Figure 3 As shown, the number of risk factor features is K, the number of fusion features is d, and the number of nodes in the graph convolutional network is N.

[0170] In embodiments and other possible embodiments of this disclosure, the concatenated features are input into the Graph Convolutional Network (GCN) to achieve COPD identification and / or classification. The GCN is a method for deep learning on graph data. Essentially, a Graph Convolutional Neural Network functions similarly to a Convolutional Neural Network (CNN)—it's a feature extractor, but its object is graph data. The GCN ingeniously designs a method for extracting features from graph data, allowing us to use these features for node classification, graph classification, and link prediction, and incidentally obtain graph embeddings, demonstrating its wide range of applications.

[0171] In this disclosure, the method for identifying and / or classifying COPD based on the graph convolutional network and utilizing the splicing features includes: obtaining the number of nodes N of the graph convolutional network; and based on the splicing features V... (l)Based on the risk factor characteristics, the sub-edge constraint matrix corresponding to the number of risk factor characteristics is obtained; and the sub-edge constraint matrices are fused to obtain the edge constraint matrix E; based on the splicing feature V corresponding to the number of nodes N. (l) The fusion features are used to obtain a three-dimensional difference matrix; and a convolution operation is performed on the three-dimensional difference matrix to obtain an edge weight matrix W; an adjacency matrix is ​​obtained based on the edge constraint matrix E and the edge weight matrix W; the adjacency matrix and the splicing feature V are then used to obtain the edge weight matrix W. (l) To achieve the identification and / or classification of COPD.

[0172] In embodiments and other possible embodiments of this disclosure, the splicing feature V (l) This includes: K-dimensional risk factor features (vector) and d-dimensional fusion features (vector).

[0173] In this disclosure, the method for obtaining a sub-margin constraint matrix corresponding to the number of risk factor features based on the risk factor features of the spliced ​​features includes: arranging the risk factor features of each spliced ​​feature in columns to obtain the number of risk factor feature vectors; taking one feature of each risk factor feature vector as a base feature, and performing element-wise difference operations between all features of each risk factor feature vector and the base feature to obtain the sub-margin constraint matrix corresponding to the number of risk factor features.

[0174] In embodiments and other possible embodiments of this disclosure, for example, K=4, the number is 4, therefore the risk factor features of each of the spliced ​​features are arranged in columns to obtain 4 risk factor feature vectors (scale N×1); one feature in the risk factor feature vector is used as the base feature, and the remaining features in the risk factor feature vector are subtracted from the base feature to obtain an N×1 risk factor feature vector, wherein the first element in the risk factor feature vector is 0 (base feature minus itself); similarly, N-1 features in the risk factor feature vector are used as base features to obtain an N×(N-1) risk factor feature vector; the risk factor feature vectors are spliced ​​to obtain a sub-edge constraint matrix of size N×N. Therefore, the risk factor features of the 4 spliced ​​features, after the above operations, yield K=4 sub-edge constraint matrices of size N×N.

[0175] In this disclosure, the method for fusing the sub-edge constraint matrices to obtain an edge constraint matrix includes: performing element-wise addition on the sub-edge constraint matrices to obtain the edge constraint matrix.

[0176] In the embodiments of this disclosure and other possible embodiments, for example, K=4, the corresponding elements of the four N×N sub-edge constraint matrices are added together and then standardized to obtain the edge constraint matrix.

[0177] In this disclosure, the method for obtaining a three-dimensional difference matrix based on the fusion features of the spliced ​​features corresponding to the number of nodes includes: arranging the fusion features of each spliced ​​feature in columns to obtain a fusion feature vector; copying the number of fusion feature vectors and performing a splicing operation to obtain a three-dimensional feature; transposing the three-dimensional feature to obtain a transposed three-dimensional feature; performing element-wise difference operations on the three-dimensional feature and the transposed three-dimensional feature to obtain a three-dimensional difference matrix.

[0178] In the embodiments and other possible embodiments disclosed herein, the fused features of each spliced ​​feature are arranged in columns to obtain a fused feature vector with a scale of N×106; the number of fused feature vectors are copied N-1 times to obtain N N×106 fused feature vectors; the N N×106 fused feature vectors are spliced ​​to obtain a three-dimensional feature with a scale of N×N×106; the three-dimensional feature with a scale of N×N×106 is transposed to obtain a transposed three-dimensional feature; the three-dimensional feature and the transposed three-dimensional feature are subjected to element-wise difference operations and absolute value operations to obtain a three-dimensional difference matrix with a scale of N×N×106.

[0179] The 106 fused feature vectors are copied and concatenated to obtain three-dimensional features. These three-dimensional features are then transposed to obtain transposed three-dimensional features. Element-wise subtraction is performed on the three-dimensional features and the transposed three-dimensional features to obtain a three-dimensional difference matrix. Specifically, these 106 fused features are copied N-1 times and combined with the original N×106 fused features to form an N×N×106 three-dimensional matrix. The Z-axis remains unchanged, and the matrix is ​​transposed along the X and Y axes, resulting in another N×N×106 three-dimensional matrix. Subtracting these two matrices yields an N×N×106 three-dimensional difference matrix. This subtraction operation ensures that the 106 features of different nodes are subtracted one by one to obtain a three-dimensional adjacency matrix.

[0180] This disclosure proposes a novel method for performing a convolution operation on the three-dimensional difference matrix to obtain an edge weight matrix. Specifically, the method for performing a convolution operation on the three-dimensional difference matrix to obtain an edge weight matrix includes: using a convolution kernel larger than 1×1 to perform an upsampling operation on the three-dimensional difference matrix to obtain an upsampled feature map; and using a convolution kernel larger than 1×1 to perform a downsampling operation on the upsampled feature map to obtain an edge weight matrix.

[0181] Figure 4 A schematic diagram of a convolutional network structure for performing convolution operations using a three-dimensional difference matrix according to an embodiment of this disclosure is shown. Figure 4 As shown, the proposed convolutional network for the 3D difference matrix is ​​a ∩ structure, and can therefore be defined as ∩Net. ∩Net transforms the N × N × 10⁶ 3D difference matrix into an N × N edge weight matrix. A 3 × 3 convolution kernel is used to upsample the N × N × 10⁶ 3D difference matrix to obtain an upsampled feature map; a 3 × 3 convolution kernel is then used to downsample the upsampled feature map; finally, a 1 × 1 convolution kernel is used to convolve the feature map to obtain an N × N × 1 edge weight matrix.

[0182] Specifically, if a 1×1 convolutional kernel is directly used to superimpose different channels, the elements do not affect each other. This results in the output matrix lacking relationships between differences, meaning some effective information is not utilized. Differences between different nodes are indeed related, and failing to utilize these relationships during training degrades model performance. Therefore, a 3×3 convolutional kernel is introduced to extract this information; this process is equivalent to reshaping an adjacency matrix. An upsampling followed by downsampling operation is employed, so the matrix after the network not only contains relationships between nodes but also relationships between differences between different groups of nodes. Furthermore, residual connections are introduced into the network. Residual connections are highly effective in deepening networks, allowing a smaller amount of data to be used in a deeper network to achieve a better structure. ∩Net's network is already deep, and adding residual connections enhances the network's effectiveness.

[0183] In this disclosure, the method for obtaining an adjacency matrix based on the edge constraint matrix and the edge weight matrix includes: performing element-wise multiplication of the edge constraint matrix and the edge weight matrix to obtain the adjacency matrix.

[0184] In this disclosure, the method for identifying and / or classifying COPD based on the graph convolutional network and utilizing the adjacency matrix and the spliced ​​features includes: performing matrix multiplication on the adjacency matrix and the spliced ​​features to obtain an adjacency feature matrix; passing the adjacency feature matrix through a fully connected layer (FC) to obtain an adjacency feature vector Gn(V(l) ); and the adjacent feature vector Gn(V (l) ) and the splicing feature V (l) By concatenating the data, we obtain the classification vector (V). (l+1) The classification vector is then used to identify and / or classify COPD.

[0185] Specifically, the edge weight matrix W and the edge constraint matrix E are multiplied element-wise to obtain the adjacency matrix used in the graph network; the adjacency matrix is ​​then concatenated with the input node features V. (l) Matrix multiplication is performed, which updates the node matrix. After the update, the result Gn(V) is obtained by passing the fully connected layer. (l) ), and then in the splicing feature V of this layer (l) By concatenating the sequences, we obtain the result for this layer. In the last layer, Gn(V) (last) Instead of being concatenated with the input nodes, the output of the unknown nodes is directly input into the softmax layer, and the output is normalized to obtain the final prediction of the unknown nodes.

[0186] In another embodiment and other possible embodiments of this disclosure, the method for identifying or classifying COPD (chronic obstructive pulmonary disease) includes: acquiring a lung image to be processed and a preset recognition model; projecting the lung image to be processed at multiple preset angles to obtain multiple projected images; and using the multiple projected images based on the preset recognition model to complete the identification or classification of COPD in the lung image to be processed.

[0187] Acquire the lung image to be processed and the preset recognition model.

[0188] In the embodiments and other possible embodiments disclosed herein, the preset recognition model can be a machine learning-based classification model, such as one or more of Support Vector Machine (SVM), Multilayer Perceptron (MLP), Random Forest (RF), K-Nearest Neighbors (KNN), Logistic Regression (LR), Decision Tree (DT), Gradient Boosting (GB), and Linear Discriminant Analysis (LDA). The preset recognition model can also be a deep learning classification model, such as the ResNet network. See details below. Figure 5 Detailed explanation.

[0189] Figure 5 A deep learning-based classification model according to an embodiment of this disclosure is shown. Figure 5As shown, conv represents a convolutional layer, s represents stride, pool represents a pooling layer, avg pool represents an average pooling layer, fc represents a fully connected layer, COPD represents a person diagnosed with COPD, and HC represents a person diagnosed without COPD (healthy person). Specifically, the deep learning-based classification model includes: four modules; each module is repeated twice, with shortcut connections between modules. Each module mainly contains convolutional layers, with conv kernels having either 3×3 or 1×1 values. The network ends with an avg pool layer and a fully connected fc layer. The main difference between ResNet26 and ResNet50 is the number of modules. ResNet26 has (2,2,2,2) modules, while ResNet50 has (3,4,6,3) modules. The main difference between ResNet26 and ResNet26d is in the downsampling operation within each module. The ResNet26 network has a downsampling operation in each module, such as... Figure 5 As shown, there are two paths. The left path has three convolutional layers with kernels of 1×1, 3×3, and 1×1, and kernel strides of 2, 1, and 1, respectively. The right path has one convolutional layer with a 1×1 kernel and a kernel stride of 2. The main difference between ResNet26d and ResNet26 is that the kernel strides in the left path are 1, 2, and 1, respectively. In this disclosure, transfer learning is an effective method for ResNet-26d to avoid overfitting. This study uses ResNet convolutional weights from a pre-trained ImageNet model and trains the fully connected layers and output layer using the dataset presented in this study.

[0190] The lung image to be processed is projected at multiple preset angles to obtain multiple projected images.

[0191] In the embodiments of this disclosure and other possible embodiments, the set multiple angles can be at least one set angle, such as any angle corresponding to the cross-section, coronal plane, or sagittal plane of the lung image to be processed. That is, the lung image to be processed can be an image in the xyz plane, which can be projected according to the three set angles (angle in the x direction, angle in the y direction, and angle in the z direction) of the xyz plane to obtain multiple projected images (projected image of the cross-section, projected image of the coronal plane, and projected image of the sagittal plane). At the same time, this disclosure does not limit the set multiple angles. For example, the set multiple angles can also be angles of 30°, 45°, 60°, etc. in the xy plane, yz plane, x direction, y direction, and z direction of the xz plane.

[0192] In this disclosure, the method for projecting the lung image to be processed according to a set multiple angles to obtain multiple projected images includes: acquiring a set multiple angles and projection criteria; and projecting the lung image to be processed according to the set multiple angles and the projection criteria respectively to obtain multiple projected images.

[0193] In the embodiments of this disclosure and other possible embodiments, the projection criterion can be the maximum intensity projection (MIP) criterion or the minimum intensity projection criterion, or it can be a projection criterion for a certain density interval. For example, the lung image to be processed can be projected according to the maximum intensity projection criterion along the z-axis direction, taking the maximum value, to obtain the corresponding projected image. Simultaneously, the lung image to be processed can also be projected according to the maximum intensity projection criterion along the x-axis and y-axis directions, respectively, taking the maximum values, to obtain two corresponding projected images.

[0194] For example, the lung image to be processed is a CT image, which has three layers of xy images, namely... Projecting the image by taking the maximum value along the z-axis yields the corresponding projected image.

[0195] For example, the lung image to be processed is a CT image, which has three layers of xy images, respectively. Projecting the image by taking the minimum value along the z-axis yields the corresponding projected image.

[0196] Meanwhile, this disclosure proposes a density interval projection criterion to extract a set density interval for projection, obtaining the projection corresponding to the density of interest, thereby improving COPD identification. Specifically, the method of projecting the lung image to be processed according to the density interval projection criterion from multiple set angles to obtain multiple projected images includes: selecting values ​​of the lung image to be processed within a set density interval according to the multiple set angles, and superimposing the values ​​within the set density interval to obtain multiple projected images.

[0197] For example, the lung image to be processed is a CT image, which has three layers of xy images, namely the first xy image. Second xy graph Third xy image Projecting along the z-axis within a defined density range [1,3] yields the corresponding projected image. Specifically, the 2 in the first row and first column of the projected image is obtained by adding the 1 in the first row and first column of the first xy image within the set density interval [1,3]. Since the 5 in the first row and first column of the third xy image is not within the set density interval [1,3], the 5 is not added to other values.

[0198] Based on the preset recognition model, COPD identification is completed using the multiple projected images to process the lung image to be processed.

[0199] In this disclosure, the method for identifying or classifying COPD in a lung image to be processed using the multiple projected images based on the preset identification or classification model includes: inputting the multiple projected images into the preset identification or classification model to obtain multiple first classification results; statistically analyzing the results of having COPD and not having COPD in the multiple first classification results to obtain a first statistical result of having COPD and a second statistical result of not having COPD; if the first statistical result is greater than the second statistical result, it is determined that the person has COPD; otherwise, it is determined that the person does not have COPD.

[0200] For example, in embodiments of this disclosure and other possible embodiments, three projection images (a cross-sectional projection image, a coronal projection image, and a sagittal projection image) are input respectively. Figure 5 In the preset identification model, the preset identification model inputs three corresponding first classification results; the results of having COPD and not having COPD in the three first classification results are statistically analyzed to obtain a first statistical result (2) of having COPD and a second statistical result (1) of not having COPD; at this time, if the first statistical result (2) is greater than the second statistical result (1), then it is determined that the person has COPD.

[0201] In the embodiments of this disclosure and other possible embodiments, before projecting the lung image to be processed at multiple angles to obtain multiple projected images, the method further includes: performing lung parenchyma segmentation on the lung image to be processed to obtain a lung parenchyma image; projecting the lung parenchyma image at multiple angles to obtain multiple projected images; and then, based on the preset recognition model, using the multiple projected images, completing the COPD recognition of the lung image to be processed.

[0202] In embodiments of this disclosure and other possible embodiments, a method for performing lung parenchyma segmentation on the lung image to be processed to obtain a lung parenchyma image includes: acquiring a lung segmentation model; and performing lung parenchyma segmentation on the lung image to be processed based on the lung segmentation model to obtain a lung parenchyma image. The segmentation model can be a lung segmentation model based on traditional segmentation algorithms or a lung segmentation model based on deep learning, such as a lung segmentation model based on U-Net or U-ResNet. The training methods for these models are techniques commonly used by those skilled in the art, and will not be described in detail here.

[0203] In this disclosure, before projecting the lung image to be processed at multiple angles to obtain multiple projected images, the method further includes: performing lung parenchyma segmentation on the lung image to be processed to obtain a lung parenchyma image; calculating preset radiomics features corresponding to the lung parenchyma image based on a radiomics computing model; and using the multiple projected images and the preset radiomics features based on the preset recognition model to complete the COPD identification of the lung image to be processed.

[0204] In embodiments of this disclosure and other possible embodiments, a method for performing lung parenchyma segmentation on the lung image to be processed to obtain a lung parenchyma image includes: acquiring a lung segmentation model; and performing lung parenchyma segmentation on the lung image to be processed based on the lung segmentation model to obtain a lung parenchyma image. The segmentation model can be a lung segmentation model based on a traditional segmentation algorithm, or a lung segmentation model based on deep learning, such as a lung segmentation model based on U-Net or U-ResNet.

[0205] In this disclosure, a method for identifying COPD in a lung image to be processed, based on the preset recognition model and utilizing the multiple projected images and the preset radiomics features, includes: obtaining a preset convolutional neural network; using the preset convolutional neural network to extract features from the multiple projected images respectively to obtain multiple sets of convolutional features; fusing the multiple sets of convolutional features and the preset radiomics features to obtain recognition features; and using the recognition features based on the preset recognition model to identify COPD in the lung image to be processed.

[0206] In embodiments of this disclosure and other possible embodiments, the preset convolutional neural network may be selected. Figure 5The publicly available model extracts features from the multiple projected images to obtain multiple sets of convolutional features. Alternatively, the segmentation model proposed in the paper *Med3d: Transfer learning for 3D medical image analysis* (Chen, S., K. Ma and Y. Zheng) can be selected; however, only the encoding structure 3D ResNet10, 3D ResNet18, or 3D ResNet34 within the segmentation model needs to be used to extract the convolutional features of the lung parenchyma image. Then, the multiple sets of convolutional features and the preset radiomics features are fused to obtain recognition features. Based on one or more of the preset recognition models, such as Support Vector Machine (SVM), Multilayer Perceptron (MLP), Random Forest (RF), K-Nearest Neighbors (KNN), Logistic Regression (LR), Decision Tree (DT), Gradient Boosting (GB), and Linear Discriminant Analysis (LDA), the recognition features are used to complete the COPD identification of the lung image to be processed.

[0207] This disclosure proposes a method for fusing convolutional features and radiomics features to improve the accuracy of COPD identification. Specifically, the method for fusing the multiple sets of convolutional features and the preset radiomics features to obtain identification features includes: fusing the radiomics features with each set of convolutional features of the multiple sets of convolutional features to obtain multiple sets of identification features; and using the preset identification model to complete the COPD identification of the lung image to be processed.

[0208] In the embodiments of this disclosure and other possible embodiments, the method of fusing the radiomics features with each of the multiple sets of convolutional features to obtain multiple sets of recognition features includes: performing a concatenation operation on the radiomics features and each of the multiple sets of convolutional features to obtain multiple sets of recognition features.

[0209] In the embodiments of this disclosure and other possible embodiments, the radiomics features and the multiple convolutional features are both radiomics features and multiple convolutional features corresponding to the segmented lung parenchyma image.

[0210] For example, the multiple sets of convolutional features corresponding to multiple projection images (cross-sectional projection image, coronal projection image, sagittal projection image) of the lung image to be processed or multiple projection images (cross-sectional projection image, coronal projection image, sagittal projection image) of the lung parenchyma image after lung segmentation are respectively: the first set of convolutional features is [1, 2, 3], the second set of convolutional features is [4, 5, 6], the third set of convolutional features is [7, 8, 9], and the radiomics feature is [1, 1, 1]. The radiomics feature and each set of convolutional features of the multiple sets of convolutional features are concatenated to obtain multiple sets of recognition features: [1, 2, 3, 1, 1, 1], [4, 5, 6, 1, 1, 1], [7, 8, 9, 1, 1, 1]. Wherein, the radiomics feature and the multiple sets of convolutional features are the same radiomics feature and multiple sets of convolutional features corresponding to the same lung image to be processed.

[0211] In the embodiments and other possible embodiments of this disclosure, whether COPD identification of the lung image to be processed is completed based on the preset recognition model using the multiple projected images, or based on the preset recognition model using the multiple sets of recognition features respectively, it is necessary to first train the preset recognition model, and then complete the COPD identification of the lung image to be processed based on the trained preset recognition model. Meanwhile, the method for training the preset recognition model is a technical means commonly used by those skilled in the art, and will not be described in detail in this disclosure.

[0212] Based on the above, in this disclosure, the method for identifying COPD in the lung image to be processed by utilizing the multiple sets of identification features based on the preset identification model includes: inputting the multiple sets of identification features into the preset identification model to obtain multiple second classification results; statistically analyzing the results of having COPD and not having COPD in the multiple second classification results to obtain a third statistical result of having COPD and a fourth statistical result of not having COPD; if the third statistical result is greater than the fourth statistical result, it is determined that the person has COPD; otherwise, it is determined that the person does not have COPD.

[0213] In the embodiments of this disclosure and other possible embodiments, the multiple sets of convolutional features and preset radiomics features corresponding to the three projection images (transverse projection image, coronal projection image, and sagittal projection image) are respectively stitched together to obtain three sets of recognition features; the three sets of recognition features are input into a preset recognition model, and the preset recognition model is input with three corresponding second classification results; the results of having COPD and not having COPD in the three second classification results are statistically analyzed to obtain a third statistical result (2 results) of having COPD and a fourth statistical result (1 result) of not having COPD; if the third statistical result (2 results) is greater than the fourth statistical result (1 result), then it is determined that the person has COPD.

[0214] In embodiments of this disclosure, the lung image includes: an inspiratory lung image and an expiratory lung image; a registration operation is performed on the inspiratory lung image and the expiratory lung image to obtain displacement parameters corresponding to the transition from inhalation to exhalation; the set local features, global features, and displacement parameters are fused to obtain fused features.

[0215] In the embodiments of this disclosure, before performing the registration operation of the inspiratory lung image and the expiratory lung image, lung region segmentation is performed on the inspiratory lung image and the expiratory lung image respectively to obtain corresponding inspiratory lung region images and expiratory lung region images; the registration operation of the inspiratory lung region and the expiratory lung region is performed to obtain the displacement parameters corresponding to the inspiratory to expiratory phases.

[0216] In embodiments of this disclosure and other possible embodiments, the registration algorithm may use an elastic registration algorithm or utilize a VGG network (VGG-net) in deep learning, as described in the paper "Deformable image registration using convolutional neural networks" or a U-net, as described in the paper "Pulmonary CT Registration through Supervised Learning with Convolutional Neural Networks". This disclosure does not limit the specific registration algorithm.

[0217] In the embodiments of this disclosure and other possible embodiments, the displacement parameter corresponding to the inhalation to exhalation includes at least the movement distance, which can be calculated using the Euclidean distance formula.

[0218] In embodiments of this disclosure, the method for fusing the set local features, global features, and displacement parameters to obtain fused features includes: splicing the set local features, global features, and displacement parameters to obtain fused features.

[0219] Step S103: Based on the set classifier, the fusion features are used to complete the identification or classification of the patient's breathing difficulties and / or grading.

[0220] In embodiments of this disclosure and other possible embodiments, the defined classifier may be one or more of Support Vector Machine (SVM), Multilayer Perceptron (MLP), Random Forest (RF), K-Nearest Neighbors (KNN), Logistic Regression (LR), Decision Tree (DT), Gradient Boosting (GB), Linear Discriminant Analysis (LDA), etc. The defined classifier is trained using fused features obtained by fusing local and global features of multiple lung function data and / or lung image corresponding image features.

[0221] The entity executing the breathing difficulty analysis method can be a breathing difficulty analysis device. For example, the breathing difficulty analysis method can be executed by a terminal device, server, or other processing device. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, the breathing difficulty analysis method can be implemented by a processor calling computer-readable instructions stored in memory.

[0222] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0223] The entity executing the breathing difficulty analysis method can be a breathing difficulty analysis device. For example, the breathing difficulty analysis method can be executed by a terminal device, server, or other processing device. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, the breathing difficulty analysis method can be implemented by a processor calling computer-readable instructions stored in memory.

[0224] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0225] Meanwhile, this disclosure also discloses an analysis device for dyspnea, the analysis device comprising: an acquisition unit for acquiring pulmonary function data and / or image features corresponding to lung images of a patient to be analyzed; a fusion unit for selecting corresponding set local features from the pulmonary function data and / or the image features, determining global features based on the pulmonary function data and / or the image features, and fusing the set local features and global features to obtain fused features; and an analysis unit for identifying or classifying the patient's dyspnea based on a set classifier and using the fused features.

[0226] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0227] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0228] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured as described above. The electronic device can be provided as a terminal, a server, or other type of device.

[0229] Figure 6This is a block diagram illustrating an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, or other terminal.

[0230] Reference Figure 6 The electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0231] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0232] Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0233] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0234] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0235] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0236] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0237] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 can detect the on / off state of electronic device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0238] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0239] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0240] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions that can be executed by a processor 820 of an electronic device 800 to perform the above-described method.

[0241] Figure 7 This is a block diagram illustrating an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. (Refer to...) Figure 7 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0242] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output (I / O) interface 1958. Electronic device 1900 can operate on an operating system stored in memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0243] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0244] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0245] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0246] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0247] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions to implement various aspects of this disclosure.

[0248] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0249] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0250] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0251] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0252] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method of analyzing dyspnea, characterized by, include: If the patient to be analyzed has the chronic obstructive pulmonary disease (COPD) or the COPD of the patient to be analyzed reaches the set grade, then the patient to be analyzed will be subjected to dyspnea analysis. The process involves: acquiring lung function data and / or imaging features corresponding to lung images of the patient to be analyzed; prior to acquiring these features, identifying and / or grading chronic obstructive pulmonary disease (COPD) in the patient, including: acquiring lung images of the patient to be analyzed; segmenting the lung images to obtain lung parenchymal images; extracting omics features and convolutional features from the lung parenchymal images; performing feature selection on the omics features and convolutional features based on COPD identification or grading labels corresponding to the lung images to obtain selected radiomics features and selected convolutional features; performing a first fusion operation on the selected radiomics features and selected convolutional features to obtain a first fusion feature; and identifying and / or grading COPD based on the first fusion feature and a preset classifier; or, acquiring lung images of the patient to be analyzed; segmenting the lung images to obtain lung parenchymal images; extracting lung parenchymal features and / or convolutional features from the lung images; and performing omics features and / or convolutional features on the lung parenchymal images. The method includes: 1) obtaining radiomics features corresponding to lung parenchymal images; 2) selecting radiomics features based on identification and / or grading labels corresponding to lung parenchymal images; 3) determining the number of radiomics features; 4) generating a radiomics feature map using the radiomics features, and performing a convolution operation on the radiomics feature map to obtain the number of convolutional features; 5) fusing the radiomics features and the convolutional features to obtain fused features; 6) determining risk factor features for a graph convolutional network based on the fused features; 7) stitching the risk factor features and the fused features to obtain stitched features; 8) using the stitched features to identify and / or grade COPD based on the graph convolutional network; or 9) acquiring lung images to be processed from the patient to be analyzed and a preset identification model; 10) projecting the lung images to be processed from multiple angles to obtain multiple projected images; 11) using the multiple projected images based on the preset identification model to complete COPD identification or grading of the lung images to be processed. Select corresponding local features from the lung function data and / or the image features corresponding to the lung images of the patient to be analyzed; determine global features based on the lung function data and / or the image features corresponding to the lung images of the patient to be analyzed; fuse the selected local features and the global features to obtain the fused features; Based on the set classifier, the fusion features are used to complete the identification or classification and / or grading of the patient's respiratory distress.

2. The analysis method according to claim 1, characterized in that, Before acquiring the lung function data corresponding to the patient to be analyzed, the lung function data is determined, including: The patient's lung function was tested using a pulmonary function testing device to obtain lung function data.

3. The method of analysis according to any one of claims 1 or 2, characterized in that, Before obtaining the imaging features corresponding to the lung image of the patient to be analyzed, the imaging features corresponding to the lung image are determined, including: Using imaging equipment, a chest image of the patient was performed to obtain a lung image; The lung image is segmented to obtain a lung region image; Feature extraction is performed on the lung region image to obtain image features.

4. The analysis method according to claim 3, characterized in that, The step of segmenting the lung image to obtain a lung region image includes: Obtain a pre-defined lung region segmentation model; Using the preset lung region segmentation model, the lung image is segmented to obtain a lung region image.

5. The analysis method according to claim 3, characterized in that, The step of extracting features from the lung region image to obtain image features includes: Obtain transfer convolutional neural networks and / or radiomics computational models; Feature extraction is performed on the lung region image using a transfer convolutional neural network and / or a radiomics computational model to obtain image features.

6. The analysis method according to claim 4, characterized in that, The step of extracting features from the lung region image to obtain image features includes: Obtain transfer convolutional neural networks and / or radiomics computational models; Feature extraction is performed on the lung region image using a transfer convolutional neural network and / or a radiomics computational model to obtain image features.

7. The assay method according to any one of claims 1, 2, 4-6, characterized in that, Before selecting corresponding predetermined local features from the lung function data and / or imaging features of the patient to be analyzed, determining the predetermined local features includes: Obtain a predefined feature selection model; based on the feature selection model, select corresponding predefined local features from multiple lung function data and / or multiple imaging features of the patient to be analyzed; or Obtain the set feature selection rules; based on the feature selection rules, select the corresponding set local features from multiple lung function data and / or multiple imaging features of the patient to be analyzed.

8. The analysis method according to claim 3, characterized in that, Before selecting corresponding predetermined local features from the lung function data and / or imaging features of the patient to be analyzed, determining the predetermined local features includes: Obtain a predefined feature selection model; based on the feature selection model, select corresponding predefined local features from multiple lung function data and / or multiple imaging features of the patient to be analyzed; or Obtain the set feature selection rules; based on the feature selection rules, select the corresponding set local features from multiple lung function data and / or multiple imaging features of the patient to be analyzed.

9. The assay method according to any one of claims 1, 2, 4-6, 8, characterized in that, The determination of global features based on the lung function data and / or imaging features of the patient to be analyzed includes: Obtain the set feature fusion model; based on the feature fusion model, perform global fusion of the lung function data and / or the image features of the patient to be analyzed to obtain the corresponding global features.

10. The analysis method according to claim 3, characterized in that, The determination of global features based on the lung function data and / or imaging features of the patient to be analyzed includes: Obtain the set feature fusion model; based on the feature fusion model, perform global fusion of the lung function data and / or the image features of the patient to be analyzed to obtain the corresponding global features.

11. The analysis method of claim 7, wherein, The determination of global features based on the lung function data and / or imaging features of the patient to be analyzed includes: Obtain the set feature fusion model; based on the feature fusion model, perform global fusion of the lung function data and / or the image features of the patient to be analyzed to obtain the corresponding global features.

12. The analytical method according to any one of claims 1, 2, 4-6, 8, 10, and 11, characterized in that, The process of fusing the defined local features and global features to obtain the fused features includes: concatenating the defined local features and global features to obtain the fused features.

13. The analysis method of claim 3, wherein, The process of fusing the defined local features and global features to obtain the fused features includes: concatenating the defined local features and global features to obtain the fused features.

14. The analysis method of claim 7, wherein, The process of fusing the defined local features and global features to obtain the fused features includes: concatenating the defined local features and global features to obtain the fused features.

15. The analysis method of claim 9, wherein, The process of fusing the defined local features and global features to obtain the fused features includes: concatenating the defined local features and global features to obtain the fused features.

16. The analytical method according to any one of claims 1, 2, 4-6, 8, 10, 11, 13-15, characterized in that, Global fusion is performed on the pulmonary function data and / or imaging features of the patient to be analyzed to obtain corresponding global features, including: Get the set contribution rate; A feature matrix is ​​constructed based on the lung function data and / or imaging features of the patient to be analyzed, and multiple feature values ​​corresponding to the feature matrix are calculated. Normalize the multiple feature values; The normalized feature values ​​are sorted, and the cumulative contribution of the sorted and normalized feature values ​​is calculated. When the cumulative contribution is greater than or equal to the set contribution rate, the feature vector of the feature value corresponding to the cumulative contribution is determined. A transformation matrix is ​​constructed based on the feature vectors; the corresponding global features are obtained based on the feature matrix and the transformation matrix.

17. The method of claim 3, wherein Global fusion is performed on the pulmonary function data and / or imaging features of the patient to be analyzed to obtain corresponding global features, including: Get the set contribution rate; A feature matrix is ​​constructed based on the lung function data and / or imaging features of the patient to be analyzed, and multiple feature values ​​corresponding to the feature matrix are calculated. Normalize the multiple feature values; The normalized feature values ​​are sorted, and the cumulative contribution of the sorted and normalized feature values ​​is calculated. When the cumulative contribution is greater than or equal to the set contribution rate, the feature vector of the feature value corresponding to the cumulative contribution is determined. A transformation matrix is ​​constructed based on the feature vectors; the corresponding global features are obtained based on the feature matrix and the transformation matrix.

18. The analysis method of claim 7, wherein, Global fusion is performed on the pulmonary function data and / or imaging features of the patient to be analyzed to obtain corresponding global features, including: Get the set contribution rate; A feature matrix is ​​constructed based on the lung function data and / or imaging features of the patient to be analyzed, and multiple feature values ​​corresponding to the feature matrix are calculated. Normalize the multiple feature values; The normalized feature values ​​are sorted, and the cumulative contribution of the sorted and normalized feature values ​​is calculated. When the cumulative contribution is greater than or equal to the set contribution rate, the feature vector of the feature value corresponding to the cumulative contribution is determined. A transformation matrix is ​​constructed based on the feature vectors; the corresponding global features are obtained based on the feature matrix and the transformation matrix.

19. The analysis method of claim 9, wherein, Global fusion is performed on the pulmonary function data and / or imaging features of the patient to be analyzed to obtain corresponding global features, including: Get the set contribution rate; A feature matrix is ​​constructed based on the lung function data and / or imaging features of the patient to be analyzed, and multiple feature values ​​corresponding to the feature matrix are calculated. Normalize the multiple feature values; The normalized feature values ​​are sorted, and the cumulative contribution of the sorted and normalized feature values ​​is calculated. When the cumulative contribution is greater than or equal to the set contribution rate, the feature vector of the feature value corresponding to the cumulative contribution is determined. A transformation matrix is ​​constructed based on the feature vectors; the corresponding global features are obtained based on the feature matrix and the transformation matrix.

20. The analysis method of claim 12, wherein, Global fusion is performed on the pulmonary function data and / or imaging features of the patient to be analyzed to obtain corresponding global features, including: Get the set contribution rate; A feature matrix is ​​constructed based on the lung function data and / or imaging features of the patient to be analyzed, and multiple feature values ​​corresponding to the feature matrix are calculated. Normalize the multiple feature values; The normalized feature values ​​are sorted, and the cumulative contribution of the sorted and normalized feature values ​​is calculated. When the cumulative contribution is greater than or equal to the set contribution rate, the feature vector of the feature value corresponding to the cumulative contribution is determined. A transformation matrix is ​​constructed based on the feature vectors; the corresponding global features are obtained based on the feature matrix and the transformation matrix.

21. The analytical method according to any one of claims 1, 2, 4-6, 8, 10, 11, 13-15, 17-20, characterized in that, The lung images include: inspiratory lung images and expiratory lung images; Perform the registration operation of the inspiratory and expiratory lung images to obtain the displacement parameters corresponding to the transition from inhalation to exhalation; The local features, global features, and displacement parameters are fused to obtain the fused features.

22. The method of claim 3, wherein, The lung images include: inspiratory lung images and expiratory lung images; Perform the registration operation of the inspiratory and expiratory lung images to obtain the displacement parameters corresponding to the transition from inhalation to exhalation; The local features, global features, and displacement parameters are fused to obtain the fused features.

23. The method of claim 7, wherein, The lung images include: inspiratory lung images and expiratory lung images; Perform the registration operation of the inspiratory and expiratory lung images to obtain the displacement parameters corresponding to the transition from inhalation to exhalation; The local features, global features, and displacement parameters are fused to obtain the fused features.

24. The analysis method of claim 9, wherein, The lung images include: inspiratory lung images and expiratory lung images; Perform the registration operation of the inspiratory and expiratory lung images to obtain the displacement parameters corresponding to the transition from inhalation to exhalation; The local features, global features, and displacement parameters are fused to obtain the fused features.

25. The analytical method according to claim 12, characterized in that, The lung images include: inspiratory lung images and expiratory lung images; Perform the registration operation of the inspiratory and expiratory lung images to obtain the displacement parameters corresponding to the transition from inhalation to exhalation; The local features, global features, and displacement parameters are fused to obtain the fused features.

26. The analytical method according to claim 16, characterized in that, The lung images include: inspiratory lung images and expiratory lung images; Perform the registration operation of the inspiratory and expiratory lung images to obtain the displacement parameters corresponding to the transition from inhalation to exhalation; The local features, global features, and displacement parameters are fused to obtain the fused features.

27. The analytical method according to claim 21, characterized in that, Before performing the registration operation of the inspiratory lung image and the expiratory lung image, lung region segmentation is performed on the inspiratory lung image and the expiratory lung image respectively to obtain the corresponding inspiratory lung region image and expiratory lung region image. Perform the registration operation of the inspiratory and expiratory lung regions to obtain the displacement parameters corresponding to the inspiratory and expiratory phases.

28. The analytical method according to any one of claims 22-26, characterized in that, Before performing the registration operation of the inspiratory lung image and the expiratory lung image, lung region segmentation is performed on the inspiratory lung image and the expiratory lung image respectively to obtain the corresponding inspiratory lung region image and expiratory lung region image. Perform the registration operation of the inspiratory and expiratory lung regions to obtain the displacement parameters corresponding to the inspiratory and expiratory phases.

29. The analytical method according to claim 21, characterized in that, The process of fusing the set local features, global features, and displacement parameters to obtain fused features includes: splicing the set local features, global features, and displacement parameters to obtain fused features.

30. The analytical method according to any one of claims 22-27, characterized in that, The process of fusing the set local features, global features, and displacement parameters to obtain fused features includes: splicing the set local features, global features, and displacement parameters to obtain fused features.

31. The analytical method according to claim 28, characterized in that, The process of fusing the set local features, global features, and displacement parameters to obtain fused features includes: splicing the set local features, global features, and displacement parameters to obtain fused features.

32. An analytical device for dyspnea, characterized in that, include: The acquisition unit is configured to perform dyspnea analysis on the patient to be analyzed if the patient has chronic obstructive pulmonary disease (COPD) or the COPD of the patient to be analyzed reaches a set grade; acquire pulmonary function data and / or imaging features corresponding to the lung images of the patient to be analyzed; before acquiring pulmonary function data and / or imaging features corresponding to the lung images of the patient to be analyzed, identify and / or grade COPD in the patient to be analyzed, including: acquiring lung images to be processed corresponding to the patient to be analyzed; performing lung region segmentation on the lung images to obtain lung parenchymal images; extracting omics features and convolutional features from the lung parenchymal images respectively; performing feature selection on the omics features and convolutional features based on the COPD identification or grading labels corresponding to the lung images respectively, to obtain selected radiomics features and selected convolutional features; performing a first fusion operation on the selected radiomics features and the selected convolutional features to obtain a first fusion feature; identifying and / or grading COPD based on the first fusion feature and a preset classifier; or, including: acquiring lung images to be processed corresponding to the patient to be analyzed. The method includes: obtaining lung images; segmenting the lung images to obtain lung parenchyma images; extracting radiomics features corresponding to the lung parenchyma images; selecting the radiomics features based on the identification and / or grading labels corresponding to the lung parenchyma images; determining the number of radiomics features; generating a radiomics feature map using the radiomics features; performing a convolution operation on the radiomics feature map to obtain the number of convolutional features; fusing the radiomics features and the convolutional features to obtain fused features; determining risk factor features for a graph convolutional network based on the fused features; stitching the risk factor features and the fused features to obtain stitched features; and using the stitched features to identify and / or grade COPD based on the graph convolutional network; or, acquiring the lung images to be processed from the patient to be analyzed and a preset identification model; projecting the lung images to be processed from multiple angles to obtain multiple projected images; and using the multiple projected images based on the preset identification model to complete the identification or grading of COPD in the lung images to be processed. The fusion unit is used to select corresponding set local features from the lung function data and / or the image features corresponding to the lung images of the patient to be analyzed, determine global features based on the lung function data and / or the image features corresponding to the lung images of the patient to be analyzed, and fuse the set local features and global features to obtain fused features. The analysis unit is used to identify and / or classify the respiratory distress of the patient to be analyzed based on the fusion features using a set classifier.

33. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method for analyzing dyspnea as described in any one of claims 1 to 31.

34. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method for analyzing respiratory distress as described in any one of claims 1 to 31.

Images