COPD phenotype determination method and apparatus, electronic device, and storage medium

Abstract

The present disclosure relates to a COPD phenotype determination method and device, electronic equipment and storage medium, and relates to the technical field of intelligent COPD phenotype prediction. The method comprises: acquiring an inhalation phase image and a plurality of set threshold intervals; using a preset airway segmentation model to perform airway segmentation on the inhalation phase image to obtain a corresponding inhalation phase airway image; using a preset synthesizer to synthesize the inhalation phase image into a corresponding synthesized exhalation phase image; determining a first parameter response map according to the inhalation phase image and the corresponding synthesized exhalation phase image and the plurality of set threshold intervals; and performing set airway elimination correction on the first parameter response map using the inhalation phase airway image or the exhalation phase airway image, and then determining a COPD phenotype. The present disclosure can realize intelligent determination of the COPD phenotype.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of intelligent diagnosis of COPD, and particularly relates to a COPD phenotype determination method and device, an electronic device and a storage medium. BACKGROUND

[0002] Chronic obstructive pulmonary disease (COPD) is the third most common cause of morbidity and mortality worldwide. It is a chronic inflammatory disease usually associated with small airway narrowing and obstruction, destruction of lung tissue and blood vessels. COPD is often considered a smoking-induced disease, but the reality is much more complex. The number of COPD cases continues to rise, with more than 300 million deaths in 2012 (6% of global deaths).

[0003] COPD has complex pathological features, and diagnosis and treatment cannot rely solely on lung function tests. Therefore, in clinical practice, markers and diagnostic techniques other than lung function tests are needed to accurately diagnose the progression and phenotype of COPD. Phenotype is a subtype of COPD with unique prognostic and therapeutic characteristics. Quantitative analysis of computed tomography (CT) images explores airway trees, lung tissue abnormalities, and other features associated with COPD. The discovery of these CT markers helps to understand the anatomical features of the lungs, thereby preventing, diagnosing, and managing COPD.

[0004] However, visual assessment and diagnosis of COPD from CT scans are subjective, and doctors have difficulty making accurate assessments without manual extraction or examination of special image features. Parameter response maps (PRMs) and disease probability maps (DPMs) have been used to quantitatively identify small airway lesions and emphysema by registering expiratory and inspiratory CT scans and reflecting their spatial locations. PRMs and DPMs have received extensive attention due to their ability to effectively assess COPD phenotypes. Although dual-phase CT scans can effectively assess COPD, they also increase some risks.

[0005] Therefore, a novel strategy is needed to synthesize parameter response maps using only inspiratory or expiratory CT scans to calculate the proportion and spatial information of different COPD phenotypes, in order to solve the problem that single-phase (inspiratory or expiratory) CT images cannot construct parameter response maps and the accuracy of COPD phenotypes of parameter response maps needs to be further improved. SUMMARY

[0006] The present disclosure proposes a COPD phenotype determination method and device, an electronic device and a storage medium technical solution.

[0007] According to an aspect of the present disclosure, a COPD phenotype determination method is provided, comprising: acquiring an inspiratory phase image or an expiratory phase image and a plurality of set threshold intervals;

[0008] performing airway segmentation on the inhalation phase image or the exhalation phase image by using a preset airway segmentation model to obtain a corresponding inhalation phase airway image or exhalation phase airway image;

[0009] synthesizing the inhalation phase image into a corresponding synthesized exhalation phase image or synthesizing the exhalation phase image into a corresponding synthesized inhalation phase image by using a preset synthesizer;

[0010] determining a first parameter response map according to the inhalation phase image and the corresponding synthesized exhalation phase image and a plurality of set threshold intervals, or determining a second parameter response map according to the exhalation phase image and the corresponding synthesized inhalation phase image and the plurality of set threshold intervals;

[0011] performing set airway rejection correction on the first parameter response map by using the inhalation phase airway image or the exhalation phase airway image to further determine a COPD phenotype, or performing set airway rejection correction on the second parameter response map by using the inhalation phase airway image or the exhalation phase airway image to further determine a COPD phenotype.

[0012] Preferably, the method of determining the first parameter response map according to the inhalation phase image and the corresponding synthesized exhalation phase image and the plurality of set threshold intervals comprises:

[0013] performing registration operation on the inhalation phase image and the corresponding synthesized exhalation phase image to obtain a corresponding first registration image;

[0014] determining the first parameter response map based on the first registration image and the plurality of set threshold intervals; and / or,

[0015] The method of determining the second parameter response map according to the exhalation phase image and the corresponding synthesized inhalation phase image and the plurality of set threshold intervals comprises:

[0016] performing registration operation on the exhalation phase image and the corresponding synthesized inhalation phase image to obtain a corresponding second registration image;

[0017] determining the second parameter response map based on the second registration image and the plurality of set threshold intervals.

[0018] Preferably, the method of synthesizing the inhalation phase image into a corresponding synthesized exhalation phase image by using the preset synthesizer comprises:

[0019] training the preset synthesizer by using an inhalation phase image for training and its corresponding exhalation phase image;

[0020] synthesizing the inhalation phase image into a corresponding synthesized exhalation phase image based on the trained preset synthesizer; and / or,

[0021] The method for training the preset synthesizer by using the training inhalation phase image and its corresponding exhalation phase image, comprises:

[0022] using the first synthesizer G I in the preset synthesizer to synthesize the training inhalation phase image into a corresponding first synthesized exhalation phase image;

[0023] using the second synthesizer G E in the preset synthesizer to convert the first synthesized exhalation phase image into a first synthesized inhalation phase image;

[0024] calculating a cycle consistency loss between the training inhalation phase image and the first synthesized inhalation phase image;

[0025] and respectively performing convolution processing on the training inhalation phase image and the first synthesized inhalation phase image to obtain corresponding inhalation phase feature maps and first synthesized inhalation phase feature maps;

[0026] calculating a perception loss between the inhalation phase feature maps and the first synthesized inhalation phase feature maps;

[0027] and using a first preset discriminator D I to determine whether the first synthesized exhalation phase image is a real image or a synthesized image;

[0028] and calculating an adversarial loss based on the result of the first preset discriminator D I ;

[0029] based on the cycle consistency loss, the perception loss and the adversarial loss, adjusting network parameters of the first synthesizer G I and the second synthesizer G E in the preset synthesizer to complete the training of the preset synthesizer; and / or,

[0030] The method for synthesizing the inhalation phase image into a corresponding synthesized exhalation phase image by using the trained preset synthesizer, comprises:

[0031] obtaining the first synthesizer G I in the trained preset synthesizer;

[0032] based on the first synthesizer G I in the trained preset synthesizer, performing convolution processing on the inhalation phase image to synthesize the inhalation phase image into a corresponding synthesized exhalation phase image.

[0033] Preferably, the method for synthesizing the exhalation phase image into a corresponding synthesized inhalation phase image by using the preset synthesizer, comprises:

[0034] training the preset synthesizer using the expiratory phase image for training and its corresponding inspiratory phase image;

[0035] synthesizing, based on the trained preset synthesizer, the expiratory phase image into a corresponding synthesized inspiratory phase image; and / or,

[0036] The method for training the training process of the preset synthesizer using the expiratory phase image for training and its corresponding inspiratory phase image, comprising:

[0037] using a second synthesizer G E in the preset synthesizer to synthesize the expiratory phase image for training into a corresponding first synthesized inspiratory phase image;

[0038] using a first synthesizer G I in the preset synthesizer to convert the first synthesized inspiratory phase image into a first synthesized expiratory phase image;

[0039] calculating a cycle consistency loss between the expiratory phase image for training and the first synthesized expiratory phase image;

[0040] and respectively performing convolution processing on the expiratory phase image for training and the first synthesized expiratory phase image to obtain corresponding expiratory phase feature maps and first synthesized expiratory phase feature maps;

[0041] calculating a perceptual loss between the expiratory phase feature map and the first synthesized expiratory phase feature map;

[0042] and using a second preset discriminator D E to determine whether the first synthesized expiratory phase image is a real image or a synthesized image;

[0043] and calculating an adversarial loss based on the result of the second preset discriminator D E ;

[0044] based on the cycle consistency loss, the perceptual loss and the adversarial loss, adjusting network parameters of the first synthesizer G I and the second synthesizer G E in the preset synthesizer to complete the training of the preset synthesizer; and / or,

[0045] The method for synthesizing, based on the trained preset synthesizer, the expiratory phase image into a corresponding synthesized inspiratory phase image, comprising:

[0046] obtaining a second synthesizer G E in the trained preset synthesizer;

[0047] based on the trained second synthesizer G of the preset synthesizers E performing convolution processing on the expiratory phase image to synthesize the expiratory phase image into a corresponding synthetic inspiratory phase image.

[0048] Preferably, the determination method further comprises: performing lung vessel segmentation on the inspiratory phase image or the expiratory phase image by using a preset lung vessel segmentation model to obtain a corresponding inspiratory phase lung vessel image or expiratory phase lung vessel image.

[0049] Before performing the set airway exclusion correction on the first parameter response graph by using the inspiratory phase airway image or the expiratory phase airway image, performing lung vessel exclusion correction on the first parameter response graph to obtain a first corrected parameter response graph; and performing set airway exclusion correction on the parameter response graph to obtain a second corrected parameter response graph; and determining the COPD phenotype based on the second corrected parameter response graph; or, performing set airway exclusion correction on the first parameter response graph by using the inspiratory phase airway image or the expiratory phase airway image to obtain a third parameter response graph; performing lung vessel exclusion correction on the parameter response graph to obtain a fourth corrected parameter response graph; and determining the COPD phenotype based on the fourth corrected parameter response graph; and / or,

[0050] Before performing the set airway exclusion correction on the second parameter response graph by using the inspiratory phase airway image or the expiratory phase airway image, performing lung vessel exclusion correction on the second parameter response graph to obtain a fifth corrected parameter response graph; and performing set airway exclusion correction on the parameter response graph to obtain a sixth corrected parameter response graph; and determining the COPD phenotype based on the sixth corrected parameter response graph; or, performing set airway exclusion correction on the second parameter response graph by using the inspiratory phase airway image or the expiratory phase airway image to obtain a seventh parameter response graph; performing lung vessel exclusion correction on the parameter response graph to obtain an eighth corrected parameter response graph; and determining the COPD phenotype based on the eighth corrected parameter response graph.

[0051] Preferably, in the set airway exclusion correction, the method for determining the set airway comprises:

[0052] determining the corresponding pipe wall diameter of the airway affecting COPD, and determining the set airway based on the pipe wall diameter; and / or,

[0053] Before the determining the first parameter response map according to the inhalation phase image and the corresponding synthesized exhalation phase image and a plurality of set threshold intervals, or the determining the second parameter response map according to the exhalation phase image and the corresponding synthesized inhalation phase image and a plurality of set threshold intervals, a lung field segmentation model is used to perform lung field segmentation on the inhalation phase image or the exhalation phase image to obtain a corresponding inhalation phase lung field image and a corresponding synthesized exhalation phase lung field image or the exhalation phase lung field image and a corresponding synthesized inhalation phase lung field image;

[0054] Further, the first parameter response map is determined according to the inhalation phase lung field image and the corresponding synthesized exhalation phase lung field image and a plurality of set threshold intervals, or the second parameter response map is determined according to the exhalation phase lung field image and the corresponding synthesized inhalation phase lung field image and a plurality of set threshold intervals.

[0055] Preferably, the method for determining the set airway based on the tube wall diameter comprises:

[0056] The airway tube wall diameter in the inhalation phase airway image or the exhalation phase airway image is measured respectively;

[0057] The airway with an airway tube wall diameter greater than or equal to the tube wall diameter is determined as the set airway.

[0058] According to an aspect of the present disclosure, a COPD phenotype determination device is provided, comprising:

[0059] An acquisition unit is configured to acquire an inhalation phase image or an exhalation phase image and a plurality of set threshold intervals;

[0060] A segmentation unit is configured to perform airway segmentation on the inhalation phase image or the exhalation phase image by using a preset airway segmentation model to obtain a corresponding inhalation phase airway image or exhalation phase airway image;

[0061] A synthesis unit is configured to synthesize the inhalation phase image into a corresponding synthesized exhalation phase image or synthesize the exhalation phase image into a corresponding synthesized inhalation phase image by using a preset synthesizer;

[0062] A determination unit is configured to determine a first parameter response map according to the inhalation phase image and the corresponding synthesized exhalation phase image and a plurality of set threshold intervals, or determine a second parameter response map according to the exhalation phase image and the corresponding synthesized inhalation phase image and a plurality of set threshold intervals;

[0063] The modification unit is configured to perform set airway exclusion correction on the first parameter response graph by using the inspiratory phase airway image or the expiratory phase airway image, and further determine the COPD phenotype; or perform set airway exclusion correction on the second parameter response graph by using the inspiratory phase airway image or the expiratory phase airway image, and further determine the COPD phenotype.

[0064] According to an aspect of the present disclosure, an electronic device is provided, comprising:

[0065] a processor;

[0066] a memory for storing processor-executable instructions;

[0067] The processor is configured to perform the COPD phenotype determination method.

[0068] According to an aspect of the present disclosure, a computer-readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the COPD phenotype determination method.

[0069] In the embodiments of the present disclosure, a COPD phenotype determination method and device, an electronic device and a storage medium are provided to solve the problems that a single-phase (inspiratory phase or expiratory phase) CT image cannot construct a parameter response graph and the COPD phenotype accuracy of the parameter response graph needs to be further improved.

[0070] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the present disclosure.

[0071] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0072] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure and serve to explain the technical solutions of the present disclosure together with the specification.

[0073] Figure 1 A flowchart of a COPD phenotype determination method according to an embodiment of the present disclosure is shown;

[0074] Figure 2 A flowchart of a method of synthesizing the inspiratory phase image into a corresponding synthetic expiratory phase image or synthesizing the expiratory phase image into a corresponding synthetic inspiratory phase image by using a preset synthesizer according to an embodiment of the present disclosure is shown;

[0075] Figure 3 A network structure diagram corresponding to the preset synthesizer according to an embodiment of the present disclosure is shown;

[0076] Figure 4 A strategy and a process for determining a PRM according to an embodiment of the present disclosure are shown;

[0077] Figure 5 A specific flow and a network architecture corresponding to a preset blood vessel segmentation model according to an embodiment of the present disclosure are shown;

[0078] Figure 6 is a block diagram of an electronic device 800 according to an exemplary embodiment;

[0079] Figure 7 is a block diagram of an electronic device 1900 according to an exemplary embodiment. DETAILED DESCRIPTION

[0080] Various exemplary embodiments, features and aspects of the present disclosure will be explained in detail below with reference to the accompanying drawings. The same reference numerals are used to represent the same elements throughout the drawings. Although various aspects of the embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless specifically noted.

[0081] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0082] The term "and / or" used herein is only a description of association relationship of associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0083] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. Those skilled in the art should understand that the present disclosure can also be implemented without some specific details. In some examples, methods, means, elements and circuits that are well known to those skilled in the art are not described in detail, in order to highlight the main idea of the present disclosure.

[0084] It can be understood that the above-mentioned various method embodiments of the present disclosure can be combined with each other to form combined embodiments without violating the principle logic. Due to the limited space, the present disclosure will not be described again.

[0085] Further, the present disclosure also provides a COPD phenotype determination apparatus, an electronic device, a computer readable storage medium, and a program, which can be used to implement any of the COPD phenotype determination methods provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding descriptions in the method section and will not be repeated here.

[0086] Figure 1 A flow chart of a COPD phenotype determination method according to an embodiment of the present disclosure is shown in FIG. 1. As shown in FIG. 1, the COPD phenotype determination method comprises the following steps. Figure 1 As shown in FIG. 1, the COPD phenotype determination method comprises the following steps. Step S101: acquiring an inhalation phase image or an exhalation phase image and a plurality of set threshold intervals; Step S102: performing airway segmentation on the inhalation phase image or the exhalation phase image by using a preset airway segmentation model to obtain a corresponding inhalation phase airway image or an exhalation phase airway image; Step S103: synthesizing the inhalation phase image into a corresponding synthesized exhalation phase image or synthesizing the exhalation phase image into a corresponding synthesized inhalation phase image by using a preset synthesizer; Step S104: determining a first parameter response map according to the inhalation phase image and the corresponding synthesized exhalation phase image, the plurality of set threshold intervals, or determining a second parameter response map according to the exhalation phase image and the corresponding synthesized inhalation phase image, the plurality of set threshold intervals; Step S105: performing set airway rejection correction on the first parameter response map by using the inhalation phase airway image or the exhalation phase airway image, and further determining a COPD phenotype, or performing set airway rejection correction on the second parameter response map by using the inhalation phase airway image or the exhalation phase airway image, and further determining a COPD phenotype. This can solve the problems that a single phase (inhalation phase or exhalation phase) CT image cannot construct a parameter response map and the COPD phenotype accuracy of the parameter response map needs to be further improved.

[0087] Step S101: acquiring an inhalation phase image or an exhalation phase image and a plurality of set threshold intervals.

[0088] In the disclosed embodiments and other possible embodiments, the inhalation phase image or the exhalation phase image can be configured as a CT image, a DR image, an MRI image, an ultrasound image, a PET image, a CT-PET image, or other medical images. Further, the CT image, the DR image, the MRI image, the ultrasound image, the PET image, the CT-PET image, or other medical images are configured as CT images, DR images, MRI images, ultrasound images, PET images, CT-PET images, or other medical images corresponding to the chest (lungs) in the inhalation state or / and the exhalation state. In addition, the CT image, the DR image, the MRI image, the ultrasound image, the PET image, the CT-PET image, or other medical images corresponding to the chest (lungs) in the inhalation state or / and the exhalation state can be configured as CT images, DR images, MRI images, ultrasound images, PET images, CT-PET images, or other medical images corresponding to the chest (lungs) in the deep inhalation state or / and the deep exhalation state.

[0089] In the disclosed embodiments and other possible embodiments, the plurality of set threshold intervals are configured as attenuation values corresponding to the inspiratory phase images or expiratory phase images. For example, the inspiratory phase images or expiratory phase images can be configured as inspiratory CT phase images or expiratory CT phase images. At this time, the plurality of set threshold intervals corresponding to the inspiratory CT phase images or synthetic inspiratory CT phase images can be configured as one or more of greater than -950HU and / or greater than -856HU, respectively; the plurality of set threshold intervals corresponding to the expiratory CT phase images or synthetic expiratory CT phase images can be configured as one or more of greater than -950HU and / or greater than -856HU, respectively. Meanwhile, for those skilled in the art, the plurality of set threshold intervals can also be configured according to the types (CT images, DR images, MRI images, ultrasound images, PET images, CT-PET images or other medical images) corresponding to the inspiratory phase images or expiratory phase images.

[0090] In the disclosed embodiments and other possible embodiments, this study has been approved by the Ethics Committee of the First Affiliated Hospital of Guangzhou Medical University, in accordance with the ethical standards of the Declaration of Helsinki (1964) and its subsequent amendments, or similar ethical standards. All participants have obtained informed consent. From August 2017 to April 2021, 558 pairs of inspiratory and expiratory CT images were collected from the First Affiliated Hospital of Guangzhou Medical University to construct Dataset 1.

[0091] All participants underwent pulmonary function tests according to the guidelines of the American Thoracic Society and the European Respiratory Society. For those whose percentage of FVC to FEV1 was less than 0.7, an additional bronchodilator test was performed within 20 minutes after administration of 180 grams of albuterol. Participants were divided into five categories according to the results of the pulmonary function test: normal, GOLD 1-4, which represents mild to severe COPD. All participants underwent chest CT scans at maximum inhalation and exhalation, respectively, to obtain corresponding inspiratory phase images and / or expiratory phase images. Information such as gender, age, and smoking status was also recorded. Patient information and acquisition parameters are listed in Table 1. Low-dose CT scans used a Siemens device with a tube voltage of 110 kVp and a slice thickness of 1 mm. The CTDIvol was 2.21 mGy. DICOM data were collected and converted to 3DNifTI image format. The matrix size of each layer of the inspiratory phase images or expiratory phase images was 512x512, and the number of slices was 280-400.

[0092] In the disclosed embodiments and other possible embodiments, before acquiring the inhaled phase image and / or the exhaled phase image and the plurality of set threshold intervals, a set preprocessing threshold interval is acquired; the inhaled phase image and / or the exhaled phase image are preprocessed based on the set preprocessing threshold interval; then, a preset airway segmentation model is used to perform airway segmentation on the preprocessed inhaled phase image or the preprocessed exhaled phase image to obtain a corresponding inhaled phase airway image or an exhaled phase airway image; a preset synthesizer is used to synthesize the preprocessed inhaled phase image into a corresponding synthesized exhaled phase image or to synthesize the preprocessed exhaled phase image into a corresponding synthesized inhaled phase image; a first parameter response map is determined according to the preprocessed inhaled phase image and the corresponding synthesized exhaled phase image, the plurality of set threshold intervals; or, a second parameter response map is determined according to the preprocessed exhaled phase image and the corresponding synthesized inhaled phase image, the plurality of set threshold intervals; the first parameter response map is corrected by airway exclusion using the preprocessed inhaled phase airway image or the exhaled phase airway image, and then a COPD phenotype is determined; or, the second parameter response map is corrected by airway exclusion using the preprocessed inhaled phase airway image or the exhaled phase airway image, and then a COPD phenotype is determined.

[0093] For example, in the disclosed embodiments and other possible embodiments, the set preprocessing threshold interval can be configured as [-1000HU, 0HU], the Hounsfield unit (HU) value is limited in the range of [-1000HU, 0HU], the pixel value in the inhaled phase image and / or the exhaled phase image that is less than the lower limit value -1000HU of the preprocessing threshold interval is set to -1000HU, and the pixel value that is greater than the upper limit value 0HU of the preprocessing threshold interval is set to 0HU, to obtain the preprocessed inhaled phase image and / or the exhaled phase image.

[0094] In addition, the inhaled phase image or the exhaled phase image can be configured as a 2D image or a 3D image, and the 3D image can be converted into a 2D DICOM slice. Among them, 558 samples are randomly divided into a training set and a test set, wherein 449 samples contain a total of 158,455 slices for model training, and 109 samples contain 38,167 DICOM slices for testing the feasibility of the model.

[0095] At the same time, in order to verify the applicability of our model, we collected data of 62 external verification cases (data set 2). The external verification data were collected from three hospitals, using different acquisition devices, and were all conventional dose CT.

[0096] Step S102: using a preset airway segmentation model to perform airway segmentation on the inhaled phase image or the exhaled phase image to obtain a corresponding inhaled phase airway image or an exhaled phase airway image.

[0097] In the disclosed embodiments and other possible embodiments, because the airway can affect the results generated by the PRM, the airway can be extracted from the fixed image (inhalation phase image or exhalation phase image) used for the registration process. The preset airway segmentation model can use an existing two-stage 3D context transformer-based U-Net model for airway segmentation proposed by the team, which is trained using CT images. The two-stage 3D context transformer-based U-Net model uses, including two stages: initial airway segmentation and fine airway segmentation, the two-stage model shares the same subnetwork and uses different airway masks as input. Using this model, the airway of the inhalation phase image or the exhalation phase image is segmented to obtain the corresponding inhalation phase airway image or exhalation phase airway image (airway tree image).

[0098] In the disclosed embodiments and other possible embodiments, the preset airway segmentation model can also be configured based on a U-Net convolutional neural network, a nnU-Net convolutional neural network, or an improved preset airway segmentation model thereof.

[0099] Step S103: using a preset synthesizer to synthesize the inhalation phase image into a corresponding synthetic exhalation phase image or to synthesize the exhalation phase image into a corresponding synthetic inhalation phase image.

[0100] In the disclosed embodiments, the method of using a preset synthesizer to synthesize the inhalation phase image into a corresponding synthetic exhalation phase image includes: training the preset synthesizer using the training inhalation phase image and its corresponding exhalation phase image; and synthesizing the inhalation phase image into a corresponding synthetic exhalation phase image based on the trained preset synthesizer.

[0101] In the disclosed embodiments, the training process of using the training inhalation phase image and its corresponding exhalation phase image to train the preset synthesizer includes: using a first synthesizer G I in the preset synthesizer to synthesize the training inhalation phase image into a corresponding first synthetic exhalation phase image; using a second synthesizer G E in the preset synthesizer to convert the first synthetic exhalation phase image into a first synthetic inhalation phase image; calculating the cycle consistency loss between the training inhalation phase image and the first synthetic inhalation phase image; and performing convolution processing on the training inhalation phase image and the first synthetic inhalation phase image, respectively, to obtain corresponding inhalation phase feature maps and first synthetic inhalation phase feature maps; calculating the perception loss between the inhalation phase feature maps and the first synthetic inhalation phase feature maps; and using a first preset discriminator D I, determine whether the first synthetic expiratory phase image is a real image or a synthetic image; and calculate an adversarial loss based on a result of the second preset discriminator D I ; adjust network parameters of the first synthesizer G I and the second synthesizer G E in the preset synthesizer based on the cycle consistency loss, the perceptual loss, and the adversarial loss, to complete training of the preset synthesizer.

[0102] In the disclosed embodiments, the method of synthesizing the inhalation phase image into a corresponding synthetic expiratory phase image based on the trained preset synthesizer includes: obtaining the first synthesizer G I in the trained preset synthesizer; and performing convolution processing on the inhalation phase image based on the first synthesizer G I in the trained preset synthesizer to synthesize the inhalation phase image into a corresponding synthetic expiratory phase image.

[0103] Similarly, in the disclosed embodiments, the method of synthesizing the expiratory phase image into a corresponding synthetic inhalation phase image using the preset synthesizer includes: training the preset synthesizer using the expiratory phase image for training and its corresponding inhalation phase image; and synthesizing the expiratory phase image into a corresponding synthetic inhalation phase image based on the trained preset synthesizer.

[0104] In the disclosed embodiments, the method of training the preset synthesizer using the expiratory phase image for training and its corresponding inhalation phase image includes: synthesizing the expiratory phase image for training into a corresponding first synthetic inhalation phase image using the second synthesizer G E in the preset synthesizer; converting the first synthetic inhalation phase image into a first synthetic expiratory phase image using the first synthesizer G I in the preset synthesizer; calculating a cycle consistency loss between the expiratory phase image for training and the first synthetic expiratory phase image; and performing convolution processing on the expiratory phase image for training and the first synthetic expiratory phase image respectively to obtain a corresponding expiratory phase feature map and a first synthetic expiratory phase feature map; calculating a perceptual loss between the expiratory phase feature map and the first synthetic expiratory phase feature map; determining whether the first synthetic expiratory phase image is a real image or a synthetic image using a second preset discriminator D E ; and calculating an adversarial loss based on a result of the second preset discriminator D E ; adjusting network parameters of the first synthesizer G I and the second synthesizer G Enetwork parameters, and complete the training of the preset synthesizer.

[0105] In the disclosed embodiments, the method for synthesizing the expiratory phase image into a corresponding synthetic inspiratory phase image based on the trained preset synthesizer includes: obtaining a second synthesizer G E in the trained preset synthesizer E , and performing convolution processing on the expiratory phase image to synthesize the expiratory phase image into a corresponding synthetic inspiratory phase image.

[0106] Figure 2 A flowchart of a method for synthesizing the inspiratory phase image into a corresponding synthetic expiratory phase image or synthesizing the expiratory phase image into a corresponding synthetic inspiratory phase image using a preset synthesizer according to an embodiment of the present disclosure is shown. As shown in Figure 2 , in the disclosed embodiments and other possible embodiments, a CycleGAN-based visual loss image synthesizer named PCycleGAN Figure 2 is proposed. PCycleGAN is based on two generators and two discriminators G I (the first synthesizer or generator), G E (the second synthesizer or generator), D I (the first preset discriminator), and D E (the second preset discriminator). Wherein, I represents the expiratory phase image, and E represents the expiratory phase image; wherein, G I learns the mapping from the inspiratory phase CT to the expiratory phase CT, and the second generator G E learns the mapping from the expiratory phase CT to the inspiratory phase CT. The first discriminator D I and the second discriminator D E are responsible for determining whether an image is a real inspiratory phase or a real expiratory phase CT image. The second discriminator D E attempts to distinguish whether the first synthesizer G I (I) is a real expiratory phase CT. Here, the embodiments of the present disclosure introduce an adversarial loss. Wherein, the adversarial loss aims to make the generator generate or synthesize high-quality CT images (synthetic expiratory phase images or synthetic inspiratory phase images) to deceive the discriminator. Wherein, the first synthesizer G I generates a fake expiratory CT image (synthetic expiratory phase image) G I (I), and then inputs it into the second generator G E to generate a fake inspiratory CT image (synthetic inspiratory phase image) G E (G I(I)), which resembles a real inspiration phase CT. Among them, the cycle consistency loss is calculated to ensure that the generated image can be reliably recovered from the source image. To obtain the perceptual loss, the inspiration phase image I and G E (G I (I)) is input into a convolutional model (e.g., a VGG network or other convolutional neural network model, etc.), and the difference between high-dimensional features is calculated. This process also applies in the opposite direction Figure 2 The lower part of (I).

[0107] In the disclosed embodiments and other possible embodiments, the key to the success of GANs is an adversarial loss that makes the generated photo indistinguishable from the real target image. The function of the loss is as follows:

[0108] L GAN (G,D Y ,X,Y)=E Y [logD Y (y)]+E X [log(1-D Y (G(x)))] (1)

[0109] Where E represents the expected value. As can be seen, D attempts to maximize the adversarial loss, while G attempts to minimize it. To further increase the stability of training, we replace the negative log-likelihood cost of the adversarial loss with a square loss function (Equation 2). Where X represents the inspiration phase CT image or the synthesized inspiration phase CT image I, and Y represents the expiration phase CT image or the synthesized expiration phase CT image.

[0110] L GAN (S,G,X,Y)=-E Y [(D(y)-1) 2 ]-E X [D(G(x) 2 ] (2)

[0111] In the disclosed embodiments and other possible embodiments, although the network can map the same input image to any image in the target domain, the adversarial loss cannot guarantee that a single input x i can be accurately mapped to y i . Since there are no paired images in training, an important loss function (Equation 3) is introduced in the CycleGAN framework to ensure consistency between input and output images: cycle consistency loss.

[0112] L cyc (G X ,G Y )=E X [||G Y (GX (x))-x||1]+E Y [||G X (G Y (y))-y||1] (3)

[0113] In the disclosed embodiments and other possible embodiments, CycleGAN also introduces an additional loss function (discrimination loss function, formula 4) to ensure that the generator does not arbitrarily change the color of the input image, and to preserve the useful features of the real samples in the target domain by taking the real samples in the target domain as the input of the generator.

[0114] L idt (G X ,G Y )=E X [||G Y (x)-x||1]+E Y [||G X (y)-y||1] (4)

[0115] In formulas (3) and (4), |||1 represents the 1-norm.

[0116] The introduction of the discrimination loss or perception loss corresponding loss function L idt to ensure that there is no unwanted change between the inhalation and exhalation CT images, and to ensure that there is no change between the inhalation and exhalation CT images (between the inhalation phase image and the corresponding synthesized exhalation phase image, between the exhalation phase image and the corresponding synthesized inhalation phase image).

[0117] In the disclosed embodiments and other possible embodiments, the introduction of the perception loss function can effectively improve the visual quality and texture features of the generated images. Compared with the loss function that compares the difference between the pixels, the perception loss function extracts high-dimensional information from the image to show the high-dimensional feature difference between the real and output images. It can be assumed that there are some flow field features in the lung CT that cannot be obtained through the pixel-level loss function. The perception loss function more effectively shows the differences and trends between different volumes. These representations are usually obtained by inputting the image into a pre-trained VGG network. After processing through multiple convolutions and other operations, the size of the image is reduced, while the dimension is increased, thereby representing new features. The formula is as follows:

[0118] L perc (G X ,G Y ,X)=E X,Y [||V(G Y (G X (x)))-V(x)||1] (5)

[0119] In formula (5), V represents a convolution model, for example, the convolution model adopts a VGG network model or other convolution neural network model.

[0120] Based on formulas (2), (3), (4) and (5), the model of the embodiment of the present disclosure is named PCycleGAN, and its loss function is defined as follows:

[0121] L PCycleGAN = L GAN (D Y ,G X ,X,Y) + L GAN (D Y ,G Y ,Y,X) + λL CYC (G X ,G Y )

[0122] + L idt (G X ,G Y ) + βL perc (G X ,G Y ,X) + βL perc (G Y ,G x ,Y) (6)

[0123] In formula (6), the first coefficient λ = 0.2 and the second coefficient β = 1. In addition, the first coefficient and the second coefficient can be configured according to actual needs by those skilled in the art.

[0124] Figure 3 A network structure diagram corresponding to a preset synthesizer (PCycleGAN) according to the embodiment of the present disclosure is shown. As shown in Figure 3 , the preset synthesizer includes two generators and two discriminators G I (the first synthesizer or generator), G E (the second synthesizer or generator), D I (the first preset discriminator), and D E (the second preset discriminator); wherein, Figure 3 (a) represents the first synthesizer or generator and / or the second synthesizer or generator; Figure 3 (a) represents D I (the first preset discriminator) and / or D E (the second preset discriminator); Figure 3 (c) represents a convolution model (for example, VGG16).

[0125] In Figure 3In the formula, the real image represents the inhalation phase image or the exhalation phase image; the synthetic image represents the exhalation phase image or the synthetic inhalation phase image; the feature map represents that the inhalation phase image for training and the first synthetic inhalation phase image are subjected to convolution processing to obtain corresponding inhalation phase feature maps and first synthetic inhalation phase feature maps, or the inhalation phase image for training and the first synthetic inhalation phase image are subjected to convolution processing, respectively, to obtain corresponding exhalation phase feature maps and first synthetic exhalation phase feature maps.

[0126] In the formula, k in convolution Cov and transposed convolution TransposedCov represents the number of convolution kernels, and s represents the convolution step length. Figure 3 In the formula, k in convolution Cov and transposed convolution TransposedCov represents the number of convolution kernels, and s represents the convolution step length.

[0127] In the formula, k in convolution Cov and transposed convolution TransposedCov represents the number of convolution kernels, and s represents the convolution step length. Figure 3 In the formula, k in convolution Cov and transposed convolution TransposedCov represents the number of convolution kernels, and s represents the convolution step length. Figure 3 As shown in FIG. 8 (a), the network coding-decoding structure used is as follows: the input is a corresponding inhalation phase image or exhalation phase image of a 512x512 2D array with C channels. C can be set to 1, which means that the input is a single-channel array. The encoder is composed of a 4x4 convolution layer with a step length of 2, followed by six groups of LeakyReLU layers, a combination of a 4x4 convolution layer with a step length of 2 and a normalization layer, and a combination of a 4x4 convolution layer with a step length of 2 and a LeakyReLU layer. The decoder is composed of seven groups of ReLU layers, a combination of a 4x4 transposed convolution layer with a step length of 2 and a normalization layer, and a combination of a 4x4 up-convolution layer with a step length of 2 and a Tanh activation function. During the down-sampling process, the feature map size is halved, and the number of channels is increased after each module.

[0128] In the embodiments and other possible embodiments of the present disclosure, a Markov PatchGAN discriminator can be used to distinguish between real images and false images and provide adversarial feedback. PatchGAN attempts to classify image patches. Figure 3(b) shows a PatchGAN with five convolutional layers, where the input channel is configurable to 1 and the output channel is configurable to 1. For an input image, it generates a patch, each patch point corresponding to a region in the input image. Compared with the results of PixelGAN, using PatchGAN as the discriminator has advantages in generating image details and capturing features, can reduce the number of parameters by classifying each pixel, and is easier to train.

[0129] In order to consider perceptual loss, both the generator and the real image can be input into the VGG network to extract their high-dimensional feature maps, and then the loss is calculated. Further, a VGG-16 model pre-trained on the ImageNet dataset can be used, which is commonly used for classification tasks. The network structure is shown in (c). Among them, the disclosure uses the part before the second pooling layer in the VGG-16 model, including the first 9 convolutional layers, each of which is composed of two 3x3 convolutional layers and ReLU layers, followed by a max pooling layer, and then two 3x3 convolutional layers and ReLU layers. Among them, the input image has 1 channel, and the output image has 128 channels. Figure 5

[0130] Step S104: determining a first parameter response map according to the inhalation phase image and the corresponding synthetic exhalation phase image, and a plurality of set threshold intervals; or determining a second parameter response map according to the exhalation phase image and the corresponding synthetic inhalation phase image, and a plurality of set threshold intervals.

[0131] In the disclosed embodiment, the method for determining the first parameter response map according to the inhalation phase image and the corresponding synthetic exhalation phase image, and a plurality of set threshold intervals, comprises: performing a registration operation on the inhalation phase image and the corresponding synthetic exhalation phase image to obtain a corresponding first registration image (pair); determining a first parameter response map (first PRM) based on the first registration image (pair) and the plurality of set threshold intervals.

[0132] In the disclosed embodiment, the method for determining the second parameter response map according to the exhalation phase image and the corresponding synthetic inhalation phase image, and a plurality of set threshold intervals, comprises: performing a registration operation on the exhalation phase image and the corresponding synthetic inhalation phase image to obtain a corresponding second registration image (pair); determining a second parameter response map (second PRM) based on the second registration image (pair) and the plurality of set threshold intervals.

[0133] ​In the disclosed embodiments and other possible embodiments, the method of determining a first parameter response map (first PRM) based on the first registered image and the plurality of set threshold intervals includes: performing a double-threshold operation on the inhalation phase image and the corresponding synthesized exhalation phase image (first registered image) based on the plurality of set threshold intervals, respectively, to determine the first parameter response map (first PRM). Meanwhile, the method of determining a second parameter response map (second PRM) based on the second registered image (opposite) and the plurality of set threshold intervals includes: performing a double-threshold operation on the exhalation phase image and the corresponding synthesized inhalation phase image (second registered image) based on the plurality of set threshold intervals, respectively, to determine the second parameter response map (second PRM).

[0134] For example, by setting different plurality of set threshold intervals for the inhalation and exhalation phase CT images, and based on the two registered images (the inhalation phase image and the corresponding synthesized exhalation phase image; or, the exhalation phase image and the corresponding synthesized inhalation phase image), each position in the two registered images can be classified into four categories, thereby determining the first parameter response map (first PRM) and / or the second parameter response map (second PRM). Each category corresponds to a phenotype of COPD, which are emphysema region, functional small airway to disease (fSAD) region, no feature region, and normal region (no emphysema and fSAD region), respectively; wherein the emphysema region, the fSAD region, and the normal region are represented by red, yellow, and green, respectively. Green represents normal, with the attenuation value in the inhalation phase CT image or the synthesized inhalation phase CT image being greater than -950 HU, and the voxel value (attenuation value) in the exhalation phase CT image or the synthesized exhalation phase CT image being greater than -856 HU. Red represents emphysema, with the attenuation value in the inhalation phase CT image or the synthesized inhalation phase CT image being less than -950 HU, and the attenuation value in the exhalation phase CT image or the synthesized exhalation phase CT image being less than -856 HU. Yellow represents the fSAD region, with the attenuation value in the inhalation phase CT image or the synthesized inhalation phase CT image being greater than -950 HU, and the attenuation value in the exhalation phase CT image or the synthesized exhalation phase CT image being less than -856 HU. Further, the proportion value of each phenotype can be obtained by calculating the proportion of the number of voxels in each category to the total number of voxels in the lung field (left lung and / or right lung), thereby quantitatively analyzing COPD.

[0135] In the disclosed embodiments, before determining the first parameter response map (first PRM) based on the inhalation phase image and the corresponding synthesized exhalation phase image and a plurality of set threshold intervals, or determining the second parameter response map (second PRM) based on the exhalation phase image and the corresponding synthesized inhalation phase image and a plurality of set threshold intervals, a lung field segmentation model is used to perform lung field segmentation on the inhalation phase image or the exhalation phase image to obtain a corresponding inhalation phase lung field image and a corresponding synthesized exhalation phase lung field image, or the exhalation phase lung field image and a corresponding synthesized inhalation phase lung field image; then, the first parameter response map (first PRM) is determined based on the inhalation phase lung field image and the corresponding synthesized exhalation phase lung field image and a plurality of set threshold intervals, or the second parameter response map (second PRM) is determined based on the exhalation phase lung field image and the corresponding synthesized inhalation phase lung field image and a plurality of set threshold intervals.

[0136] In the disclosed embodiments and other possible embodiments, the method of determining the first parameter response map (first PRM) based on the first registration image and the plurality of set threshold intervals further comprises: using a lung field segmentation model to perform lung field segmentation on the inhalation phase image or the synthesized exhalation phase image to obtain a first lung field image; and performing double-threshold operation on the inhalation phase image and the corresponding synthesized exhalation phase image (first registration image) based on a plurality of set threshold intervals in the first lung field image to determine the first parameter response map (first PRM) corresponding to the first lung field image.

[0137] Meanwhile, in the disclosed embodiments and other possible embodiments, the method of determining the second parameter response map (second PRM) based on the second registration image and the plurality of set threshold intervals further comprises: using a lung field segmentation model to perform lung field segmentation on the exhalation phase image or the synthesized inhalation phase image to obtain a second lung field image; and performing double-threshold operation on the exhalation phase image and the corresponding synthesized inhalation phase image (second registration image) based on a plurality of set threshold intervals in the second lung field image to determine the second parameter response map (second PRM) corresponding to the second lung field image.

[0138] In the disclosed embodiments and other possible embodiments, the lung field segmentation model is used to segment the lung field of the inspiratory phase image or the synthetic expiratory phase image to obtain a first lung field image, or the lung field segmentation model is used to segment the lung field of the expiratory phase image or the synthetic inspiratory phase image to obtain a second lung field image, which includes a lung segmentation (lung field segmentation) and a labeling operation. The lung segmentation is to exclude the influence of external factors on the generated image effect and facilitate the subsequent registration process. Hofmanninger et al. proposed a model for segmenting the lung region and obtaining a lung label, which is used for the labeling operation to extract the lung region (lung field region) from the original image (the inspiratory phase image or the synthetic expiratory phase image or the expiratory phase image or the synthetic inspiratory phase image).

[0139] In the disclosed embodiments and other possible embodiments, the lung field segmentation model can also be configured as a lung field segmentation model based on a U-Net convolutional neural network, a nnU-Net convolutional neural network, or an improved lung field segmentation model thereof.

[0140] In addition, in the disclosed embodiments and other possible embodiments, we also use Imbio to determine the COPD phenotype from the dual-gas phase CT images (inspiratory phase image and expiratory phase image). In the test set, Imbio determined the COPD phenotype for 109 cases and compared it with our method.

[0141] In the disclosed embodiments and other possible embodiments, the registration process is performed using the Elastix tool. The registration mainly includes two steps, the first is affine transformation, which allows translation, rotation, scaling and tilting of the image to be registered (floating image), and the second is non-rigid B-spline transformation. The B-spline transformation is modeled as a weighted sum of B-spline basis functions, which are placed on a uniform grid of control points; at the same time, the B-spline basis functions have local support, which is beneficial for fast computation. Among them, the image to be registered (moving image) can be configured as a synthetic expiratory phase image or a synthetic inspiratory phase image, and the fixed image is configured as an inspiratory phase image or an expiratory phase image. Similarly, the image to be registered (floating image) can be configured as an inspiratory phase image or an expiratory phase image, and the fixed image is configured as a synthetic expiratory phase image or a synthetic inspiratory phase image.

[0142] In the disclosed embodiments and other possible embodiments, other existing registration algorithms can also be selected by those skilled in the art, such as the SIFT registration algorithm, to implement the registration operation of the inspiratory phase image and the corresponding synthetic expiratory phase image or the registration operation of the expiratory phase image and the corresponding synthetic inspiratory phase image.

[0143] Step S105: performing a set airway elimination correction on the first parameter response graph by using the inspiratory phase airway image or the expiratory phase airway image, and then determining the COPD phenotype; or performing a set airway elimination correction on the second parameter response graph by using the inspiratory phase airway image or the expiratory phase airway image, and then determining the COPD phenotype.

[0144] In the disclosed embodiments, the determination method further comprises: performing lung vessel segmentation on the inspiratory phase image or the expiratory phase image by using a preset vessel segmentation model to obtain a corresponding inspiratory phase lung vessel image or expiratory phase lung vessel image; before performing the set airway elimination correction on the first parameter response graph by using the inspiratory phase airway image or the expiratory phase airway image, performing a lung vessel elimination correction on the first parameter response graph to obtain a first corrected parameter response graph; and performing a set airway elimination correction on the parameter response graph to obtain a second corrected parameter response graph; and determining the COPD phenotype based on the second corrected parameter response graph; or performing a set airway elimination correction on the first parameter response graph by using the inspiratory phase airway image or the expiratory phase airway image to obtain a third parameter response graph; performing a lung vessel elimination correction on the parameter response graph to obtain a fourth corrected parameter response graph; and determining the COPD phenotype based on the fourth corrected parameter response graph.

[0145] In the disclosed embodiments, the determination method further comprises: before performing the set airway elimination correction on the second parameter response graph by using the inspiratory phase airway image or the expiratory phase airway image, performing a lung vessel elimination correction on the second parameter response graph to obtain a fifth corrected parameter response graph; and performing a set airway elimination correction on the parameter response graph to obtain a sixth corrected parameter response graph; and determining the COPD phenotype based on the sixth corrected parameter response graph; or performing a set airway elimination correction on the second parameter response graph by using the inspiratory phase airway image or the expiratory phase airway image to obtain a seventh parameter response graph; performing a lung vessel elimination correction on the parameter response graph to obtain an eighth corrected parameter response graph; and determining the COPD phenotype based on the eighth corrected parameter response graph.

[0146] In the disclosed embodiments and other possible embodiments, the preset vessel segmentation model has preset vessel intervals; the inspiratory phase lung vessel image or the expiratory phase lung vessel image is obtained by performing lung vessel segmentation on the inspiratory phase image or the expiratory phase image by using the preset vessel intervals; or the inspiratory phase lung vessel image or the expiratory phase lung vessel image is obtained by performing lung vessel segmentation on the synthesized inspiratory phase image or the synthesized expiratory phase image by using the preset vessel intervals.

[0147] In the disclosed embodiments and other possible embodiments, the preset blood vessel interval can be configured as [-500HU, -1000HU], and those skilled in the art can configure the preset blood vessel interval according to needs. Specifically, before the double-threshold operation, the registered CT image and the fixed image are operated to limit the threshold to [-500HU, -1000HU], and the voxel value exceeding -500HU is set to 0. Because the blood vessels can affect the synthesis of PRM, and the attenuation value of most blood vessels is between -500HU and 0HU, the influence of lung blood vessels on the determination of PRM is reduced.

[0148] In the disclosed embodiments and other possible embodiments, the preset blood vessel segmentation model can also be configured as a preset blood vessel segmentation model based on a U-Net convolutional neural network, a nnU-Net convolutional neural network, or an improved preset blood vessel segmentation model.

[0149] Meanwhile, in the disclosed embodiments and other possible embodiments, other preset airway segmentation models can be used. For example, the present disclosure proposes a preset airway or preset blood vessel segmentation model based on semi-supervised learning, which includes: training a first segmentation model using a first number of first lung images and their corresponding lung blood vessel label images; and performing lung blood vessel segmentation on a second number of second lung images based on the trained first segmentation model to obtain corresponding first lung blood vessel segmentation images; selecting the corresponding first lung blood vessel segmentation images to obtain selected first lung blood vessel segmentation images; and training a second segmentation model using the first number of first lung images and their corresponding lung blood vessel label images, the selected first lung blood vessel segmentation images, and their corresponding second lung images; and performing lung blood vessel segmentation on the remaining third number of second lung images after selection based on the trained second segmentation model to obtain corresponding second lung blood vessel segmentation images. This solves the problem that the structure of the lung blood vessel tree is complex, the diameter size is not uniform, and there are many bifurcations, especially in the labeling of small blood vessels, which makes it very difficult to label data, resulting in a small amount of labeled data and low quality, thereby providing a basis for accurate determination of COPD phenotypes.

[0150] In the disclosed embodiments and other possible embodiments, the first lung image and the second lung image can be configured as CT images, DR images, MRI images, ultrasound images, PET images, CT-PET images, or other medical images. Further, the CT images, DR images, MRI images, ultrasound images, PET images, CT-PET images, or other medical images can be configured as CT images, DR images, MRI images, ultrasound images, PET images, CT-PET images, or other medical images corresponding to the chest (lungs) in the inhalation state or / and the exhalation state. In addition, the CT images, DR images, MRI images, ultrasound images, PET images, CT-PET images, or other medical images corresponding to the chest (lungs) in the inhalation state or / and the exhalation state can be configured as CT images, DR images, MRI images, ultrasound images, PET images, CT-PET images, or other medical images corresponding to the chest (lungs) in the deep inhalation or / and the deep exhalation.

[0151] In the disclosed embodiments, before the first segmentation model is trained using the first number of first lung images and the corresponding lung blood vessel label images thereof, the method further comprises: obtaining the first number of first lung images and the corresponding lung blood vessel label images thereof, and a second number of second lung images.

[0152] In the disclosed embodiments, before the first segmentation model is trained using the first number of first lung images and the corresponding lung blood vessel label images thereof, the method further comprises: performing lung field segmentation on the first number of first lung images and the second number of second lung images respectively, to obtain corresponding first lung field images and second lung field images; further, training the first segmentation model using the first number of first lung field images and the corresponding lung blood vessel label images thereof; and performing lung blood vessel segmentation on the second number of second lung field images based on the trained first segmentation model, to obtain corresponding first lung blood vessel segmentation images; selecting the corresponding first lung blood vessel segmentation images to obtain selected first lung blood vessel segmentation images; and training the second segmentation model using the first number of first lung field images and the corresponding lung blood vessel label images thereof, the selected first lung blood vessel segmentation images, and the corresponding second lung field images thereof; and performing lung blood vessel segmentation on the remaining third number of second lung field images after selection based on the trained second segmentation model, to obtain corresponding second lung blood vessel segmentation images.

[0153] In the disclosed embodiments and other possible embodiments, the method of performing lung field segmentation on the first number of first lung images and the second number of second lung images respectively to obtain corresponding first lung field images and second lung field images comprises: obtaining a preset lung field segmentation model, and performing lung field segmentation on the first number of first lung images and the second number of second lung images respectively by using the preset lung field segmentation model to obtain corresponding first lung field images and second lung field images. At the same time, after performing lung field segmentation on the first number of first lung images and the second number of second lung images respectively to obtain corresponding first lung field images and second lung field images, the first lung images and the second lung images in all subsequent processing methods are configured as corresponding first lung field images and second lung field images respectively.

[0154] In the disclosed embodiments and other possible embodiments, the preset lung field segmentation model can be configured based on a U-Net convolutional neural network, or a UNETR convolutional neural network, or a SwinUNETR convolutional neural network, or a nnU-Net convolutional neural network, or an improved preset lung field segmentation model thereof.

[0155] In the disclosed embodiments, a first segmentation model is trained by using a first number of first lung images and corresponding lung vessel label images thereof; and lung vessel segmentation is performed on a second number of second lung images based on the trained first segmentation model to obtain corresponding first lung vessel segmentation images.

[0156] For example, the first number is configured as 12 cases, and the second number is configured as 168 cases. That is, 12 cases of first lung images have lung vessel label images; and 168 cases of second lung images do not have lung vessel label images.

[0157] In the disclosed embodiments and other possible embodiments, the first lung images of 12 cases are from the data set provided by the VESSEL12 (https: / / vessel12.grand-challenge.org / ) challenge. The challenge provides some chest CT scans for download. Each downloaded file contains CT scans stored in Meta (or MHD / RAW) format. This format stores images as an ASCII-readable header file with the extension.mhd and a separate binary file of image data with the extension.raw. Among them, about half of the scan images of the first lung images of 12 cases contain abnormalities such as emphysema, nodules or pulmonary embolism, and the maximum slice interval is 1 millimeter. Three scans with labels are downloaded. For each lung region in these scans, the mask and vascular annotation csv files are included in the download. Each annotation CSV file contains a list of labeled points for a single scan. Each point has been labeled independently by three annotators. Only points on which the three annotators agree are included. The annotation file is provided in csv format. The format of each point is "x, y, z, label", where (x, y, z) represents the position information of the label in each first lung image, and the voxel with the label configured as 1 represents a blood vessel, and the voxel with the label configured as 0 is classified as a non-blood vessel (pulmonary vessel label image corresponding to the first lung image of 12 cases).

[0158] In the disclosed embodiments and other possible embodiments, the second lung images of 168 cases are from the First Affiliated Hospital of Guangzhou Medical University, configured as non-enhanced CT images, and the thickness of all CT scans is 1.0 mm, the slice size is 512x512, and the storage format is dicom.

[0159] In the disclosed embodiments and other possible embodiments, Figure 5 The specific process and network architecture corresponding to the preset blood vessel segmentation model according to the embodiments of the present disclosure are shown, Figure 5 In the disclosed embodiments and other possible embodiments, The first segmentation model (Teacher Model, teacher model) is fully supervised trained by using the first lung images of the first number (12 cases) and the corresponding pulmonary vessel label images, to obtain an initial teacher model (the first segmentation model after training of the teacher model). Then, Figure 5 In the disclosed embodiments and other possible embodiments, Based on the trained first segmentation model, the pulmonary blood vessels of the second lung images of the second number (168 cases) are segmented to obtain the corresponding first pulmonary blood vessel segmentation images (168 cases of first pseudo labels).

[0160] The embodiments of the present disclosure are fully supervised trained using 12 cases of lung CT scan data and corresponding labels (the first segmentation model is trained using the first number of first lung images and their corresponding lung vessel label images), and 10 cases of data are used for testing. First, the lung area (lung field) is automatically segmented from each CT image (first lung image) to obtain the corresponding first lung field image, and the gold standard of the blood vessels in the lung area (lung vessel label image) is obtained. In order to solve the problem of class imbalance, the data within the minimum bounding box of the lung area in the first lung field image is retained.

[0161] In the embodiments of the present disclosure, the first number of first lung images or first lung field images are respectively cropped to a set size. For example, the set size of 12 cases of training data after cropping is (234, 269, 336), (301, 266, 343), (320, 235, 357), (238, 277, 323), (315, 270, 383), (270, 239, 345), (250, 259, 368), (282, 264, 403), (323, 286, 426), (296, 295, 418), (286, 254, 372), (314, 258, 399), and then the first lung images or first lung field images cropped to the set size are resampled to the median inter-voxel spacing (1x0.74x0.74mm3) of all first lung images or first lung field images. Then, the first lung images or first lung field images have a set cuboid size of 128x112x160 and are used to train the network. The number of training rounds can be initially set to 1000, and the number of input samples processed by the network at a time is 2 during each training iteration. The initial learning rate can be set to 0.01, the optimizer can use the SGD optimizer, the momentum can be set to 0.99, and the weight decay can be set to 3e-5. After the fully supervised training is completed, a teacher model is obtained, which is used for subsequent semi-supervised iterative training.

[0162] In the embodiments of the present disclosure, Figure 5 The corresponding first lung vessel segmentation images are selected to obtain selected first lung vessel segmentation images; and a second segmentation model is trained using the first number of first lung images and their corresponding lung vessel label images, the selected first lung vessel segmentation images and their corresponding second lung images; and based on the trained second segmentation model, lung vessel segmentation is performed on the remaining third number of second lung images after selection to obtain corresponding second lung vessel segmentation images. The above process is repeated until the performance indicator of the final segmentation model is higher than or equal to the set performance indicator.

[0163] ​In the disclosed embodiments and other possible embodiments, the corresponding first lung vessel segmentation images are selected to obtain selected first lung vessel segmentation images (for example, 40 first pseudo-labels); and the first number of first lung images and their corresponding lung vessel label images (12 examples), the selected first lung vessel segmentation images and their corresponding second lung images (40 examples) are used to train a second segmentation model (Student Model); and based on the trained second segmentation model (Student becomes the new teacher), the remaining third number of second lung images after selection are segmented for lung vessels to obtain corresponding second lung vessel segmentation images.

[0164] In the disclosed embodiments and other possible embodiments, the first segmentation model and / or the second segmentation model can be configured as a preset blood vessel segmentation model based on a U-Net convolutional neural network, or a UNETR convolutional neural network, or a Swin UNETR convolutional neural network, or a nnU-Net convolutional neural network, or an improved preset blood vessel segmentation model.

[0165] The first segmentation model and / or the second segmentation model (teacher model and / or student model) can be configured as a preset blood vessel segmentation model as shown. Figure 4 The preset blood vessel segmentation model (network architecture) is configured as an nnFomer model. The nFormer model can mainly be composed of 3 blocks: an encoder module (encoding module), a bottleneck module (bottleneck module), and a decoder module (decoding module), which retains the U-Net structure. The nnFormer combines a hybrid model of convolution and self-attention mechanism, fully utilizes their advantages, and proposes a computationally efficient method to capture inter-slice dependencies. In the encoder of the nnFormer, a lightweight convolutional embedding layer is added to encode spatial information at the pixel level into low-level but high-resolution 3D features. Then, after the embedding block, transformer and convolution downsampling blocks are used alternately to fully mix long-term dependencies and high-level, hierarchical object concepts, thereby improving the generalization ability and robustness of the learned representation. In addition, the nnFormer introduces V-MSA to learn the representation on 3D local, and then aggregates it to produce the prediction of the whole data.

[0166] In the disclosed embodiments and other possible embodiments, the encoder module includes, in sequence or in skip connection, an embedding layer, two local self-attention layers, a down-sampling, two local self-attention layers, a down-sampling, a bottleneck module including two global self-attention layers, a down-sampling, two global self-attention layers, an up-sampling, two global self-attention layers, an up-sampling, two global self-attention layers, a decoder module including an up-sampling, two local self-attention layers, an up-sampling, two local self-attention layers, and an expending layer.

[0167] Specifically, the preset blood vessel segmentation model (nnFormer model) is respectively configured with an embedding layer (Embedding Layer) and an expending layer (Expending Layer) at the head and tail thereof, and further comprises: two local self-attention layers (Local Self-attention Layer) connected in sequence or in a skip connection manner, a down-sampling (Down-sampling), two local self-attention layers (Local Self-attention Layer), a down-sampling (Down-sampling), two global self-attention layers (Global Self-attention Layer), a down-sampling (Down-sampling), two global self-attention layers (Global Self-attention Layer), an up-sampling (Up-sampling), two global self-attention layers (Global Self-attention Layer), an up-sampling (Up-sampling), two global self-attention layers (Global Self-attention Layer), an up-sampling (Up-sampling), two local self-attention layers (Local Self-attention Layer), an up-sampling (Up-sampling), and two local self-attention layers (Local Self-attention Layer).

[0168] In the disclosed embodiment, the processing method further comprises: obtaining a performance index of the second segmentation model; if the performance index is lower than a set performance index, selecting the corresponding second lung blood vessel segmentation image to obtain a selected second lung blood vessel segmentation image; training a third segmentation model by using the first number of first lung images and the corresponding lung blood vessel label images, the selected second lung blood vessel segmentation image and the corresponding second lung image; and performing lung blood vessel segmentation on the remaining fourth number of second lung images after the selection based on the trained third segmentation model to obtain corresponding second lung blood vessel segmentation images; and repeating the above process until the performance index of the final segmentation model is higher than or equal to the set performance index.

[0169] In the disclosed embodiments and other possible embodiments, the performance indicators of the second segmentation model can be configured as one or more of the corresponding Dice values, Iou values, sensitivity (Sensitivity), precision (Precision), and other performance indicators. At the same time, the performance indicators can be configured as one or more of the corresponding set Dice values, set Iou values, set sensitivity (Sensitivity), set precision (Precision), and other set performance indicators. At the same time, the values of the set Dice values, set Iou values, set sensitivity (Sensitivity), set precision (Precision), and other set performance indicators can be configured by the person skilled in the art as needed.

[0170] In the disclosed embodiments and other possible embodiments, if the performance indicators are lower than the set performance indicators, the corresponding second lung blood vessel segmentation images are selected to obtain selected second lung blood vessel segmentation images (40 second pseudo-labels). Further, the first number of first lung images and their corresponding lung blood vessel label images (12 examples), the selected first lung blood vessel segmentation images and their corresponding second lung images (40 examples) are used to train a second segmentation model (Student Model, student model); and based on the trained second segmentation model (Student becomes the new teacher, student model is converted into a new teacher model) to segment the lung blood vessels of the remaining third number of second lung images (128 examples-40 examples=68 examples) after selection, to obtain the corresponding second lung blood vessel segmentation images.

[0171] For example, 128 of the corresponding second lung blood vessel segmentation images are selected to obtain 40 selected second lung blood vessel segmentation images; the first number (12 examples) of first lung images and their corresponding lung blood vessel label images, the 40 selected second lung blood vessel segmentation images and their corresponding second lung images are used to train a third segmentation model or retrain the first segmentation model; and based on the trained third segmentation model or the retrained first segmentation model, the fourth number (68 examples) of second lung images after selection are segmented for lung blood vessels to obtain the corresponding second lung blood vessel segmentation images; repeat the above process until the performance indicators of the final segmentation model are higher than or equal to the set performance indicators.

[0172] In the disclosed embodiments and other possible embodiments, the third segmentation model and the segmentation model in the repeated process can be configured as a preset blood vessel segmentation model based on a U-Net convolutional neural network, or a UNETR convolutional neural network, or a SwinUNETR convolutional neural network, or a nnU-Net convolutional neural network, or an improved preset blood vessel segmentation model, or a preset blood vessel segmentation model as shown in 2.

[0173] In the disclosed embodiments, the method of selecting the corresponding first lung vessel segmentation image to obtain the selected first lung vessel segmentation image comprises: obtaining a segmentation index corresponding to each first lung vessel segmentation image and a first set segmentation index; and selecting the corresponding first lung vessel segmentation image based on the segmentation index corresponding to each first lung vessel segmentation image and the first set segmentation index to obtain the selected first lung vessel segmentation image.

[0174] In the disclosed embodiments and other possible embodiments, the segmentation index can be configured as one or more of the corresponding Dice value, Iou value, sensitivity (Sensitivity), precision (Precision), and other performance indicators. Meanwhile, the first set segmentation index, the second set segmentation index, or the set segmentation index corresponding to the segmentation model in the above-mentioned process can be configured as one or more of the corresponding set Dice value, set Iou value, set sensitivity (Sensitivity), set precision (Precision), and other set performance indicators. Meanwhile, a person skilled in the art can configure the numerical values of the set Dice value, set Iou value, set sensitivity (Sensitivity), set precision (Precision), and other set performance indicators according to actual needs.

[0175] In the disclosed embodiments, the method of selecting the corresponding first lung vessel segmentation image based on the segmentation index corresponding to each first lung vessel segmentation image and the first set segmentation index to obtain the selected first lung vessel segmentation image comprises: if the segmentation index corresponding to the first lung vessel segmentation image is greater than or equal to the first set segmentation index, determining this first lung vessel segmentation image as the selected first lung vessel segmentation image.

[0176] For example, the segmentation index is configured as precision (Precision), the first set segmentation index is configured as a set precision (Precision) value, and the selected first lung vessel segmentation image is determined from the first lung vessel segmentation image in which the precision or average set precision is greater than the set precision value or average set precision value. Wherein, a person skilled in the art can configure the set precision value or average set precision value according to actual needs, for example, 0.9.

[0177] More specifically, the first iteration uses precision as the basis for selection, selects the top 40 pseudo-labels with average precision value > 0.9 (set precision value or average set precision value) as reliable pseudo-labels (selected first lung blood vessel segmentation images), so that the ratio of labels to pseudo-labels is close to 1 to 4. Finally, 12 labeled images (the first number of first lung images and their corresponding lung blood vessel label images) and 40 unlabeled images and their pseudo-labels (selected first lung blood vessel segmentation images and their corresponding second lung images) are used for retraining to obtain a student model, completing the first iteration training.

[0178] Similarly, in the disclosed embodiments, the method of selecting the corresponding second lung blood vessel segmentation image to obtain a selected second lung blood vessel segmentation image includes: obtaining a segmentation index corresponding to each second lung blood vessel segmentation image and a second set segmentation index; and selecting the corresponding second lung blood vessel segmentation image based on the segmentation index corresponding to each second lung blood vessel segmentation image and the second set segmentation index to obtain a selected second lung blood vessel segmentation image.

[0179] Similarly, in the disclosed embodiments, the method of selecting the corresponding second lung blood vessel segmentation image based on the segmentation index corresponding to each second lung blood vessel segmentation image and the second set segmentation index to obtain a selected second lung blood vessel segmentation image includes: if the segmentation index corresponding to the second lung blood vessel segmentation image is greater than or equal to the second set segmentation index, determining this second lung blood vessel segmentation image as a selected second lung blood vessel segmentation image.

[0180] In the disclosed embodiments and other possible embodiments, the segmentation index corresponding to the second lung blood vessel segmentation image can be configured as precision or average set precision and Dice value or average Dice value. At this time, the second set segmentation index corresponding to the segmentation index is configured as a set precision value or an average set precision value and a set Dice value or an average set Dice value. Among them, the set precision value or the average set precision value can be configured by those skilled in the art according to actual needs, for example, 0.95. Similarly, the set Dice value or the average set Dice value can be configured by those skilled in the art according to actual needs, for example, 0.85.

[0181] For example, the student model (second segmentation model) trained in the first iteration is taken as the teacher model (first segmentation model) to predict the second lung images of the remaining 128 cases (a third number of remaining cases), obtaining 128 pseudo labels (second lung vessel segmentation images). As in the first iteration process, the reliable pseudo labels (selected second lung vessel segmentation images) are selected based on the accuracy or average set accuracy > 0.95 (set accuracy value or average set accuracy value) and the Dice value or average Dice value > 0.85 (set Dice value or average set Dice value). 40 reliable pseudo labels (selected second lung vessel segmentation images) are selected, and the first segmentation model is retrained or the third segmentation model is trained, completing the second iteration and obtaining the best segmentation effect (preset segmentation index).

[0182] The above embodiment only performs twice iteration, and in the disclosed embodiment and other possible embodiments, multiple iterations can be performed according to the above method by those skilled in the art to obtain a more optimal segmentation effect (preset segmentation index).

[0183] In the disclosed embodiment and other possible embodiments, the strategy of selecting reliable pseudo labels (second pseudo labels or selected first lung vessel segmentation images) is added in step S102. In step S101, the teacher model is obtained after full supervision training, and the teacher model is used to predict 168 cases of data, obtaining 168 pseudo labels (selected first lung vessel segmentation images). Then, the reliable pseudo labels (selected second pseudo labels or selected first lung vessel segmentation images) are selected, and the specific method is as follows: checkpoints are saved once every 100 rounds in the full supervision training process (network parameters or network weights of the model are saved after each training), and the checkpoint with the highest validation set Dice value is saved as the Best model (best model). After training, for each unlabeled CT scan image (a second number of second lung images), the prediction results of the pseudo labels (selected first lung vessel segmentation images) and the Best model are used to calculate the evaluation index (performance index) as the selection standard. The larger the average evaluation index, the higher the coincidence of the predicted pseudo labels, that is, the more stable the pseudo labels in the training process, and the more reliable the quality. Since the segmentation task in this study is relatively fine and aims to be applied to the quantitative index analysis of COPD disease, it is necessary to ensure that the prediction result is more real blood vessels, that is, the fewer false positives, the better.

[0184] In the disclosed embodiments, the method for determining the lung blood vessel label images corresponding to the first lung images of the first number comprises: using a preset lung blood vessel convolutional segmentation model that has been trained, performing first lung blood vessel segmentation on the first lung images of the first number respectively to obtain corresponding first lung blood vessel label images; using a machine learning segmentation model, performing second lung blood vessel segmentation on the first lung images of the first number respectively to obtain corresponding second lung blood vessel label images; wherein the tube diameter of the lung blood vessels in the second lung blood vessel label images is smaller than the tube diameter of the lung blood vessels in the first lung blood vessel label images; and fusing the first lung blood vessel label images and the second lung blood vessel label images respectively to obtain the lung blood vessel label images corresponding to the first lung images of the first number.

[0185] In the disclosed embodiments, the method for using the machine learning segmentation model to perform second lung blood vessel segmentation on the first lung images of the first number respectively to obtain corresponding second lung blood vessel label images comprises: performing multi-scale representation on the first lung images of the first number respectively to obtain corresponding multi-scale lung images; extracting features from the multi-scale lung images respectively, and using a preset classifier to classify the extracted features to obtain corresponding second lung blood vessel label images.

[0186] In the disclosed embodiments and other possible embodiments, the preset lung blood vessel convolutional segmentation model that has been trained can be configured as the previously proposed lung blood vessel convolutional segmentation model CE-NC-VesselSegNet, and the coarse blood vessels are segmented using the lung blood vessel convolutional segmentation model CE-NC-VesselSegNet to obtain the first lung blood vessel label images. Meanwhile, first, the Gaussian pyramid is used to perform multi-scale representation on the first lung images of the first number respectively, and the scale is 6; then, a feature extraction model (for example, k-means) is used to extract features from the multi-scale lung images, and a preset number (for example, 34) of features are retained; then, based on the retained preset number of features, a feature vector is calculated for each pixel, and the calculated feature vector corresponding to each pixel is input into a classifier (for example, logistic regression) to perform blood vessel probability segmentation, to obtain the probability of all pixel points and segment the complete fine blood vessels to obtain the corresponding second lung blood vessel label images.

[0187] In the disclosed embodiments, the method for fusing the first lung blood vessel label images and the second lung blood vessel label images respectively to obtain the lung blood vessel label images corresponding to the first lung images of the first number comprises: performing position superposition on the first lung blood vessel label images and the second lung blood vessel label images respectively to obtain the lung blood vessel label images corresponding to the first lung images of the first number. The first lung blood vessel label images and the second lung blood vessel label images come from the same first lung image.

[0188] In the disclosed embodiments and other possible embodiments, the complete fine vessel segmentation result (the second lung vessel label image) of each of the first lung images is fused with the coarse vessels segmented from each of the first lung images by the model CE-NC-VesselSegNet. After the fusion, the coarse vessel part does not contain non-vessel tissues such as airway walls, and thus the lung vessel label image corresponding to the first number of first lung images is obtained. In addition, the lung vessel convolutional segmentation model can be configured as other existing lung vessel segmentation models according to the needs of those skilled in the art.

[0189] In the disclosed embodiments, in the process of training the segmentation model, the loss of the segmentation model is calculated, and the network parameters of the segmentation model are adjusted using the loss.

[0190] In the disclosed embodiments, the method of calculating the loss of the segmentation model includes: obtaining a Dice loss function and a cross-entropy loss function; calculating a first loss value and a second loss value of the Dice loss function and the cross-entropy loss function, respectively; configuring the sum of the first loss value and the second loss value as the loss value of the segmentation model; and / or, the method of calculating the loss function of the segmentation model further includes: setting a loss adjustment of the Dice loss function and the cross-entropy loss function; determining whether the segmentation model passes by selecting the Dice loss function and the cross-entropy loss function to calculate the loss based on the loss adjustment.

[0191] In the disclosed embodiments, the method of setting the loss adjustment of the Dice loss function and the cross-entropy loss function includes: calculating a plurality of difference values of the segmentation pixel points between the lung vessel segmentation image corresponding to the lung image to be segmented by the segmentation model and the lung vessel label image corresponding thereto; and calculating the mean value between the plurality of difference values; if the mean value is less than a set value, the segmentation model does not calculate the loss; otherwise, the segmentation model calculates the loss by selecting the Dice loss function and the cross-entropy loss function.

[0192] In the disclosed embodiments and other possible embodiments, the loss function is defined as follows:

[0193]

[0194] The disclosure adds a calculation condition (loss adjustment) in front of the Dice loss function (Dice loss) and the cross-entropy loss function (CE loss), yn represents the predicted value or segmentation value (pulmonary vessel segmentation image) of the segmentation pixel point, y^n represents the gold standard (pulmonary vessel label image) of the segmentation pixel point, and if the difference I = |yn, y^n| between the pulmonary vessel segmentation image corresponding to the lung image to be segmented by the segmentation model and the pulmonary vessel label image corresponding thereto is less than a set value T, I = 0, otherwise the set value I = 1. The set value T of the disclosure can be configured as 0.1. By adding loss adjustment, the network (segmentation model) pays more attention to difficult points, that is, the end points and bifurcation points of the pulmonary vessels, thereby improving the segmentation accuracy.

[0195] In the disclosed embodiment and other possible embodiments, the 2-time iteration process is divided into three stages in total: 1. Supervised pre-training, an initial teacher model (trained first segmentation model) is completely trained on a first number of first lung images and their corresponding pulmonary vessel label images. 2. Generate pseudo-labels, use the initial teacher model (trained first segmentation model) to segment the pulmonary vessels of all unlabeled second number of second lung images to obtain corresponding first pulmonary vessel segmentation images (first pseudo-labels). 3. Retraining. Mix the labeled images and unlabeled images and their pseudo-labels (the first number of first lung images and their corresponding pulmonary vessel label images, selected first pulmonary vessel segmentation images and their corresponding second lung images), and retrain a student model (second segmentation model) thereon, and iterate again according to the above method until the best effect is achieved.

[0196] In the disclosed embodiment, in the process of setting the airway pruning correction, the method for determining the set airway includes: determining the wall diameter of the airway corresponding to COPD, and determining the set airway based on the wall diameter.

[0197] In the disclosed embodiment and other possible embodiments, the wall diameter can be configured as 2mm, and the airway with a diameter greater than 2mm of the wall diameter is configured as the set airway. Meanwhile, those skilled in the art can also configure the wall diameter according to actual needs.

[0198] In the disclosed embodiment, the method for determining the set airway based on the wall diameter includes: respectively measuring the airway wall diameter in the inspiratory phase airway image or the expiratory phase airway image; and determining the airway with an airway wall diameter greater than or equal to the wall diameter as the set airway.

[0199] The execution subject of the COPD phenotype determination method can be a COPD phenotype determination apparatus, for example, the COPD phenotype determination method can be executed by a terminal device or a server or other processing device, wherein the terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the COPD phenotype determination method can be implemented by a processor invoking computer readable instructions stored in a memory.

[0200] Those skilled in the art can understand that, in the above COPD phenotype determination method of the specific embodiments, the writing order of the steps does not mean a strict execution order and does not constitute any limitation on the implementation process, and the specific execution order of the steps should be determined according to the functions and possible internal logic.

[0201] In addition, the disclosure embodiments also propose a COPD phenotype determination apparatus, the COPD phenotype determination apparatus comprises: an acquisition unit configured to acquire an inhalation phase image or an exhalation phase image and a plurality of set threshold intervals; a segmentation unit configured to perform airway segmentation on the inhalation phase image or the exhalation phase image by using a preset airway segmentation model to obtain a corresponding inhalation phase airway image or an exhalation phase airway image; a synthesis unit configured to synthesize the inhalation phase image into a corresponding synthesized exhalation phase image or synthesize the exhalation phase image into a corresponding synthesized inhalation phase image by using a preset synthesizer; a determination unit configured to determine a first parameter response map according to the inhalation phase image and the corresponding synthesized exhalation phase image, the plurality of set threshold intervals, or determine a second parameter response map according to the exhalation phase image and the corresponding synthesized inhalation phase image, the plurality of set threshold intervals; and a modification unit configured to perform set airway elimination correction on the first parameter response map by using the inhalation phase airway image or the exhalation phase airway image, and further determine a COPD phenotype, or perform set airway elimination correction on the second parameter response map by using the inhalation phase airway image or the exhalation phase airway image, and further determine a COPD phenotype.

[0202] In some embodiments, the apparatus provided by the disclosure embodiments has functions or contains modules that can be used to execute the methods described in the above COPD phenotype determination method embodiments, and the specific implementation can refer to the description of the COPD phenotype determination method embodiments above. For brevity, they will not be repeated here.

[0203] The embodiment of the present disclosure also provides a computer readable storage medium, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the COPD phenotype determination method. The computer readable storage medium can be a non-volatile computer readable storage medium.

[0204] The embodiment of the present disclosure also provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the COPD phenotype determination method. The electronic device can be provided as a terminal, a server or other forms of devices.

[0205] Figure 4 The strategy and process for determining PRM according to the embodiment of the present disclosure are shown. As shown in Figure 4 The present disclosure proposes a novel strategy for synthesizing parameter response maps using inspiration phase CT scans, named "InspirationOnly", to calculate the proportion and spatial information of different COPD phenotypes. First, the present disclosure designs a cross-volume synthesis method for lung CT images based on the CycleGAN architecture. The present disclosure believes that CT images are distributed on a certain flow field, in order to effectively solve the problems of image blurring, artifacts and non-uniform errors, we combine the perceptual loss function with the CycleGAN model to extract high-dimensional features and calculate the difference. As an unsupervised model, the present disclosure does not need to use paired inspiration and expiration CT images for model training. In addition, using an unsupervised model can effectively reduce the influence of registration effect in the image generation process. Using this method, the present disclosure converts the inspiration phase CT image into an expiration phase CT image, and then performs double threshold processing to generate the PRM.

[0206] As shown in Figure 6 The present disclosure compares strategies I and II (IRE and ERI) with the proposed strategy III (InspirationOnly), in which a synthesizer needs to be trained to generate expiration phase CT images from inspiration phase CT. The synthesizer takes the inspiration phase CT as input and generates the corresponding expiration phase CT image. Of course, the shape of the generated expiration phase CT image perfectly matches the inspiration image. The synthesized-expiration phase CT is used instead of the registered expiration CT image in the ERI strategy, and the other steps are the same as the ERI strategy.

[0207] The present disclosure provides the following evaluation indicators. To evaluate the quality of the synthesized image, the present disclosure measures the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM), as well as the mean absolute error (MAE) between the synthesized image and the real image. PSNR is defined as the ratio of the peak energy of the signal to the average energy of the noise. Usually, it is expressed in decibels (dB) by taking the logarithm of the ratio.

[0208]

[0209] where SSIM (Structural Similarity Index) is used to evaluate the perceptual similarity between the real image and the synthetic image.

[0210]

[0211] where μ x and μ y represent the mean of x and y, σ x and σ y represent the standard deviation of x and y, and A represents the covariance of x and y. c1, c2 and c3 are constants used to eliminate the influence of zero in the denominator.

[0212] where the disclosure calculates the mean absolute error (MAE) between the generated expiratory image and the real image to determine the quality of different models.

[0213]

[0214] The above indicators are calculated in the lung region. In addition, the paired sample t-test is used to evaluate whether there is a significant difference between each image indicator obtained by different methods, and the results obtained by our method are used as a reference.

[0215] where the intra-class correlation coefficient (ICC) is used to evaluate whether the generated proportion is consistent with the real proportion. ICC is widely used in testing and retesting, reliability analysis, and is a reliability indicator. We use the calculation method of binomial random effect and absolute consistency.

[0216]

[0217] where MS R , MS E , MS C and n represent the row mean square, error mean square, column mean square and sample size, respectively.

[0218] In order to further evaluate the consistency of the proportion determined by different phenotypes with the real proportion, Bland-Altman analysis was performed. In order to compare the relationship between different phenotypes and lung function calculated by different methods, we calculated the Spearman correlation coefficient between different phenotypes and lung function indicators.

[0219] In order to compare the relationship between different phenotypes and lung function calculated by different methods, we calculated the Spearman correlation coefficient between different phenotypes and lung function indicators.

[0220]

[0221] where d i represents x i and y i the difference in the horizontal aspect.

[0222] In this disclosure, the proposed method will be compared with other image synthesis methods. CycleGAN model, CUT model, Pix2Pix model and ResViT model are used as comparative experiments. CUT model is an image synthesis model based on unsupervised image contrast learning proposed by Park et al. It uses a multi-layer patch-based method for one-way conversion of image domains, rather than just operating on the entire image. Pix2Pix requires paired sample supervision, while our PCycleGAN and the other four models differ in that their training only requires samples from two domains (inhale and exhale CT), without the need for paired samples. The last model compared is ResViT, a model proposed by Dalmaz et al. based mainly on the Pix2Pix framework and using a Transformer as the generator. This method takes advantage of the context sensitivity of visual transformers, as well as the accuracy of convolution operations and the reality of adversarial learning. It combines the visual transformer's response to the global environment with the local positioning ability of neural networks and the reality of adversarial generation.

[0223] In addition, most of the studies on PRM do not strictly define or require the registration direction. In order to verify whether the registration direction affects the calculation of PRM, comparative experiments are conducted to compare the IRE strategy, ERI strategy and Imbio strategy.

[0224] All methods use an input image size of 512x512, a batch size of 1, and 10 training rounds. The generator and discriminator both use the Adam optimizer with an initial learning rate of 0.0002. All experiments are run using Python 3.9.5 and CentOS Linux 7.9.2009, with the models implemented using PyTorch 1.9.0. All models are trained on a workstation with an Intel(R) Xeon(R) Silver 4114 CPU, 128 GB of memory, and an NVIDIA GeForce RTX 2080 Ti GPU.

[0225] PRMs were calculated using different methods with dual-phase CT. Two registration directions (i.e. IRE and ERI strategies) were calculated using dual-phase CT scan and compared with the results calculated by Imbio software (Table 1). For emphysema, fSAD and normal ratio, the ICCs of two registration directions were all greater than 0.99, indicating that the registration direction had no effect on PRM calculation. The ICCs between two methods and Imbio method were all greater than 0.95, indicating that there was good consistency between them. Therefore, in the following experiments, the PRM calculated by our method (i.e. ERI strategy) was used as the true value.

[0226] In clinical practice, FEV1 / FVC is used to diagnose COPD, while FEV1 / pred (predicted value of lung function parameters) is used to evaluate the severity of the patient's condition. We used the different phenotypic ratios obtained by different scanning methods with dual-phase CT to correlate with lung function (Table 2). The results obtained by the proposed method were consistent with the results calculated by Imbio, with slightly higher correlation.

[0227] Table 1 ICC of different phenotypic ratios evaluated using different methods with dual-phase CT scan

[0228]

[0229] * represents p < 0.001.

[0230] Table 2 Correlation of phenotypic ratios calculated using different methods with dual-phase CT scan and lung function

[0231]

[0232] Results are represented by R, and * represents p < 0.001.

[0233] For the synthesizer performance. The proposed PCycleGAN can accurately synthesize the registered expiratory phase CT, and the SSIM, PSNR and MAE between the synthesized and real CTs reach 0.92±0.03, 26.00±2.19 dB, 109.80±36.70 HU, respectively. To demonstrate the advantage of the proposed synthesizer, we compare them with CycleGAN, CUT, Pix2Pix and ResViT in Table 3. Our PCycleGAN outperforms the four comparative methods of CycleGAN, CUT, Pix2Pix and ResViT. Paired sample t-test shows that the results of the proposed method and the four models have significant differences in SSIM, PSNR and MAE (p<0.05). Among the four comparative models, CycleGAN performs the best. Compared with our method, the MAE of CycleGAN is 3.61 HU lower, the PSNR is 0.26 dB higher, and the SSIM is similar. Among the models that need paired images for training, Pix2Pix is slightly better than ResViT, but both methods are inferior to our method, CycleGAN and CUT. In addition, we compare the influence of GOLD grade and smoking status on the performance of PCycleGAN in image synthesis (Table 4). For subjects with higher GOLD grade (severe COPD), the quality of the synthesized images is higher than that of normal subjects (SSIM, 0.94±0.02 vs. 0.90±0.03; PSNR, 27.54±1.77 vs. 24.83±2.33; MAE, 78.69±14.58 vs. 149.14±36.04). This indicates that the proposed method performs better in synthesizing images with gas retention than normal lung tissue, making it possible to use synthesized images to determine PRM for COPD patients. In addition, the MAE of smoking subjects is significantly higher than that of non-smoking subjects. One of the possible reasons is that the disease severity of smoking subjects is higher.

[0234] Table 3 Performance of our proposed PCycleGAN and compared with CycleGAN, CUT, Pix2Pix and ResViT

[0235]

[0236] * indicates that there is a significant difference between PCycleGAN and the comparative model (paired sample t-test), p<0.05.

[0237] Table 4 PCycleGAN image synthesis performance under different GOLD grades and smoking status.

[0238]

[0239] Chronic obstructive pulmonary disease (COPD) is a highly heterogeneous disease with multiple phenotypes. The parameterized reflectance map (PRM) can be generated after registering the expiratory CT to the inspiratory CT, which can calculate the distribution and proportion of various phenotypes of COPD. However, the increased radiation dose, scan time and quality control requirements, and the limitations of patient cooperation affect the practicality of PRM. This study aims to synthesize PRM by using only inspiratory phase CT scans. Therefore, this disclosure collected clinical information, dual-phase CT images, and pulmonary function parameters from 558 subjects (normal and patients with different stages of COPD) to construct dataset 1. In addition, data from 62 subjects from another hospital were collected as dataset 2 for external validation. First, two strategies, IRE (registration direction from inspiration to expiration) and ERI (registration direction from expiration to inspiration), were adopted to generate PRM from dual-phase CT scans, and the generated PRM was compared with the results of commercial software Imbio. Second, a CycleGAN with perceptual loss (PCycleGAN) was proposed and trained for synthesizing expiratory CT images after registration completion from inspiratory CT images. Third, by replacing the real expiratory CT images with the synthesized expiratory CT images, a strategy named InspirationOnly was introduced. The results showed that the intra-class correlation coefficient (ICC) of the phenotype proportions (emphysema, functional small airway disease (fSAD), and normal) between ERI and Imbio ranged from 0.954 to 0.997. The performance of the image synthesizer was superior to the state-of-the-art model, with a mean absolute error (MAE) of 109.80±36.70HU, a peak signal-to-noise ratio (PSNR) of 26.00±2.19dB, and a structural similarity (SSIM) of 0.92±0.03. Between the InspirationOnly strategy and the ERI strategy, the ICCs of the proportions of emphysema, fSAD, and normal were 0.994, 0.823, and 0.906, respectively. The correlations of the proportions of emphysema, fSAD, and normal estimated by InspirationOnly with FEV1 / FVC were -0.78, -0.67, and 0.77, respectively. In Dataset2, the ICCs between the true and determined proportions of phenotypes were 0.832, 0.784, and 0.913.

[0240] Based on the above, the present disclosure can determine PRM from paired inspiration and expiration CT using two different registration directions, namely IRE strategy and ERI strategy, and the results are consistent with those calculated by Imbio. In the case of using registered expiration CT images as the true benchmark, a PCycleGAN model using perceptual loss is proposed, and expiration CT images are synthesized from inspiration CT images by a non-paired training method. With the synthesized CT images, a strategy named InspirationOnly is proposed, which can calculate emphysema, fSAD and normal phenotype proportions, and show the pixel distribution of any single individual and the progression pattern of COPD disease, which is consistent with the results obtained by the ERI strategy. Finally, the developed model performs well in an external validation dataset including CT images acquired under regular radiation dose and different examination conditions (hospital, CT model).

[0241] Therefore, the proposed PCycleGAN can synthesize high-quality registered expiration phase CT images from inspiration phase CT, which may benefit from perceptual loss and the training method using non-paired CT slices as input. PCycleGAN enables PRM generation by the InspirationOnly strategy using only inspiration phase CT scans. The estimation of spatial distribution and proportion of COPD phenotypes is consistent with the results of dual-phase CT scans and is related to lung function parameters. At the same time, InspirationOnly also shows good generalization ability. This provides a potential tool for calculating the proportion of COPD phenotypes, especially in the absence of expiration phase CT images.

[0242] The contributions of the present disclosure are summarized as follows: 1. A strategy for parametric response map synthesis using inspiration CT scans is proposed. By introducing perceptual loss combined with CycleGAN model, the generation model is improved, which eliminates the need for paired images and reduces the influence of registration effect in model training. PRM performance is more superior than similar generation models. 2. A process of bidirectional CT registration and PRM calculation is designed, which improves the process of airway segmentation and image registration. The consistency of bidirectional registration and PRM calculated by Imbio LLC (Imbio, Minneapolis, MN) is compared. 3. The effectiveness of the proposed strategy is demonstrated by comparing different supervised and unsupervised models.

[0243] Figure 6 is a block diagram of an electronic device 800 according to an exemplary embodiment. The electronic device 800 can be, for example, a terminal such as a mobile phone, a computer, a digital broadcasting terminal, a message transmitting / receiving device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0244] Reference Signs Figure 7The electronic device 800 can include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0245] The processing component 802 usually controls overall operations of the electronic device 800, such as operations associated with displaying, making phone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to complete all or part of steps of the above methods. In addition, the processing component 802 can include one or more modules to facilitate

[0246] The memory 804 is configured to store various types of data to support operations of the electronic device 800. Examples of these data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic or optical disk.

[0247] The power supply component 806 provides power for the various components of the electronic device 800. The power supply component 806 can include a power supply management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0248] The multimedia component 808 includes a screen to provide an output interface between the electronic device 800 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and intensity of the touching or sliding action. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zooming capability.

[0249] The audio component 810 is configured to output and / or input an audio signal. For example, the audio component 810 includes a microphone (MIC) to receive an external audio signal when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker to output an audio signal.

[0250] The I / O interface 812 provides an interface for the processing component 802 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0251] The sensor component 814 includes one or more sensors to provide various state assessments for the electronic device 800. For example, the sensor component 814 can detect an open / closed state of the electronic device 800, relative positioning of components, such as a display and a keypad of the electronic device 800, a change in position of the electronic device 800 or a component of the electronic device 800, presence or absence of user contact with the electronic device 800, an orientation or acceleration / deceleration of the electronic device 800, and a temperature change of the electronic device 800. The sensor component 814 can include a proximity sensor configured to detect presence of a nearby object without any physical touch. The sensor component 814 can further include a light sensor such as a CMOS or CCD image sensor for use in an imaging application. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0252] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) techniques, infrared data association (IrDA) techniques, ultra-wideband (UWB) techniques, Bluetooth (BT) techniques, and other techniques.

[0253] In an exemplary embodiment, the electronic device 800 can be implemented with one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements, for performing the above-described methods.

[0254] In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 804 including computer program instructions, is also provided, which can be executed by the processor 820 of the electronic device 800 to complete the above-described methods.

[0255] Figure 7 is a block diagram of an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 can be provided as a server. Referring to ​ The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932, for storing instructions, such as application programs, executable by the processing component 1922. The application programs stored in the memory 1932 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described methods.

[0256] The electronic device 1900 can also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0257] In example embodiments, a non-transitory computer-readable storage medium, e.g., memory 1932 including computer program instructions, is also provided that can be executed by processing component 1922 of electronic device 1900 to implement the above-described methods.

[0258] The present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.

[0259] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a

[0260] The computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0261] Computer readable program instructions for carrying out operations of the present disclosure can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0262] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0263] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0264] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0265] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0266] Embodiments of the present disclosure have been described above, and the description is intended to be illustrative, and not restrictive, of the disclosed embodiments. Many modifications and variations of the described embodiments are possible, and all such modifications and variations are intended to be within the scope of the described embodiments. The description used herein is intended to best explain the principles of the various embodiments, the practical application, and the technical improvements over related art found in the market, or to enable others skilled in the art to understand the various embodiments disclosed herein. The terminology used herein is for the purpose of describing the various embodiments only and is not intended to be limiting.

Claims

1. A method of COPD phenotype determination, characterized in that, The method comprises: acquiring an inhalation phase image and a plurality of set threshold intervals; performing airway segmentation on the inhalation phase image by using a preset airway segmentation model to obtain a corresponding inhalation phase airway image; synthesizing the inhalation phase image into a corresponding synthetic exhalation phase image by using a preset synthesizer; wherein the step of synthesizing the inhalation phase image into a corresponding synthetic exhalation phase image by using a preset synthesizer comprises: training the preset synthesizer by using a training inhalation phase image and a corresponding exhalation phase image thereof; and synthesizing the inhalation phase image into a corresponding synthetic exhalation phase image based on the trained preset synthesizer; determining a first parameter response map according to the inhalation phase image and the corresponding synthetic exhalation phase image and the plurality of set threshold intervals; performing set airway rejection correction on the first parameter response map by using the inhalation phase airway image, and further determining a COPD phenotype.

2. The determination method according to claim 1, characterized in that, The step of determining a first parameter response map according to the inhalation phase image and the corresponding synthetic exhalation phase image and the plurality of set threshold intervals comprises: performing registration operation on the inhalation phase image and the corresponding synthetic exhalation phase image to obtain a corresponding first registration image; determining a first parameter response map based on the first registration image and the plurality of set threshold intervals.

3. The determination method according to any one of claims 1 or 2, characterized in that, The training process of training the preset synthesizer by using a training inhalation phase image and a corresponding exhalation phase image thereof comprises: using a first synthesizer G of the preset synthesizers I synthesizing the inhalation phase images for training into corresponding first synthesized exhalation phase images; using a second synthesizer G of the preset synthesizers E converting the first synthesized expiratory phase image into a first synthesized inspiratory phase image; calculating a cycle consistency loss between the training inhalation phase image and the first synthetic inhalation phase image; respectively performing convolution processing on the training inhalation phase image and the first synthetic inhalation phase image to obtain a corresponding inhalation phase feature map and a first synthetic inhalation phase feature map; calculating a perception loss between the inhalation phase feature map and the first synthetic inhalation phase feature map; using a first preset discriminator D I , determining whether the first synthesized expiratory phase image is a real image or a synthesized image; based on the first preset discriminator D I calculate the adversarial loss according to the result; Based on the cycle consistency loss, the perception loss and the adversarial loss, adjust network parameters of a first synthesizer G in the preset synthesizer I and a second synthesizer G E , to complete training of the preset synthesizer.

4. The determination method according to any one of claims 1 or 2, characterized in that, The step of synthesizing the inhalation phase image into a corresponding synthetic exhalation phase image based on the trained preset synthesizer comprises: obtaining a first synthesizer G in the preset synthesizer after training I ; based on the trained first synthesizer G in the preset synthesizer I The inspiratory phase image is convoluted to synthesize a corresponding synthesized expiratory phase image.

5. The determination method according to claim 3, characterized in that, The step of synthesizing the inhalation phase image into a corresponding synthetic exhalation phase image based on the trained preset synthesizer comprises: obtaining a first synthesizer G in the preset synthesizer after training I ; based on the trained first synthesizer G in the preset synthesizer I The inspiratory phase image is convoluted to synthesize a corresponding synthesized expiratory phase image.

6. The determination method according to any one of claims 1 or 2 or 5, characterized in that, Further comprising: performing lung blood vessel segmentation on the inhalation phase image by using a preset blood vessel segmentation model to obtain a corresponding inhalation phase lung blood vessel image; performing lung blood vessel rejection correction on the first parameter response map to obtain a first corrected parameter response map before performing set airway rejection correction on the first parameter response map by using the inhalation phase airway image; performing set airway rejection correction on the first corrected parameter response map to obtain a second corrected parameter response map; determining a COPD phenotype based on the second corrected parameter response map.

7. The determination method according to claim 3, characterized in that, Further comprising: performing lung blood vessel segmentation on the inhalation phase image by using a preset blood vessel segmentation model to obtain a corresponding inhalation phase lung blood vessel image; performing lung blood vessel rejection correction on the first parameter response map to obtain a first corrected parameter response map before performing set airway rejection correction on the first parameter response map by using the inhalation phase airway image; performing set airway rejection correction on the first corrected parameter response map to obtain a second corrected parameter response map; determining a COPD phenotype based on the second corrected parameter response map.

8. The determination method according to claim 4, characterized in that, Further comprising: The inspiratory phase image is segmented by using a preset blood vessel segmentation model to obtain a corresponding inspiratory phase lung blood vessel image; Before the set airway exclusion correction is performed on the first parameter response graph according to the inspiratory phase airway image, the first parameter response graph is corrected by lung blood vessel exclusion to obtain a first corrected parameter response graph; and the set airway exclusion correction is performed on the first corrected parameter response graph to obtain a second corrected parameter response graph. The COPD phenotype is determined based on the second corrected parameter response graph.

9. The determination method according to any one of claims 1 or 2 or 5 or 7 or 8, characterized in that, In the set airway exclusion correction, the set airway is determined by determining a tube wall diameter corresponding to an airway affecting COPD and determining the set airway based on the tube wall diameter.

10. The determination method according to claim 3, characterized in that, In the set airway exclusion correction, the set airway is determined by determining a tube wall diameter corresponding to an airway affecting COPD and determining the set airway based on the tube wall diameter.

11. The determination method according to claim 4, characterized in that, In the set airway exclusion correction, the set airway is determined by determining a tube wall diameter corresponding to an airway affecting COPD and determining the set airway based on the tube wall diameter.

12. The determination method according to claim 6, characterized in that, In the set airway exclusion correction, the set airway is determined by determining a tube wall diameter corresponding to an airway affecting COPD and determining the set airway based on the tube wall diameter.

13. The determination method of claim 9, wherein, The set airway is determined based on the tube wall diameter by measuring the airway tube wall diameter in the inspiratory phase airway image or the expiratory phase airway image respectively, and determining the airway corresponding to the airway tube wall diameter greater than or equal to the tube wall diameter as the set airway.

14. The determination method according to any one of claims 10-12, characterized in that, The set airway is determined based on the tube wall diameter by measuring the airway tube wall diameter in the inspiratory phase airway image or the expiratory phase airway image respectively, and determining the airway corresponding to the airway tube wall diameter greater than or equal to the tube wall diameter as the set airway.

15. The determination method according to any one of claims 1 or 2 or 5 or 7 or 8, 10-13, characterized in that, Before the first parameter response graph is determined according to the inspiratory phase image and the corresponding synthetic expiratory phase image and a plurality of set threshold intervals, the inspiratory phase image and the synthetic expiratory phase image are segmented by using a lung field segmentation model to obtain a corresponding inspiratory phase lung field image and a corresponding synthetic expiratory phase lung field image; The first parameter response graph is determined according to the inspiratory phase lung field image and the corresponding synthetic expiratory phase lung field image and a plurality of set threshold intervals.

16. The determination method according to claim 3, characterized in that, Before the first parameter response graph is determined according to the inspiratory phase image and the corresponding synthetic expiratory phase image and a plurality of set threshold intervals, the inspiratory phase image and the synthetic expiratory phase image are segmented by using a lung field segmentation model to obtain a corresponding inspiratory phase lung field image and a corresponding synthetic expiratory phase lung field image; The first parameter response graph is determined according to the inspiratory phase lung field image and the corresponding synthetic expiratory phase lung field image and a plurality of set threshold intervals.

17. The determination method of claim 4, wherein, Before determining the first parameter response map according to the inhalation phase image, the corresponding synthetic exhalation phase image and a plurality of set threshold intervals, the lung field segmentation model is used for lung field segmentation of the inhalation phase image and the synthetic exhalation phase image, to obtain a corresponding inhalation phase lung field image and a corresponding synthetic exhalation phase lung field image. The first parameter response map is determined according to the inhalation phase lung field image and the corresponding synthetic exhalation phase lung field image and a plurality of set threshold intervals.

18. The determination method of claim 6, wherein, Before determining the first parameter response map according to the inhalation phase image, the corresponding synthetic exhalation phase image and a plurality of set threshold intervals, the lung field segmentation model is used for lung field segmentation of the inhalation phase image and the synthetic exhalation phase image, to obtain a corresponding inhalation phase lung field image and a corresponding synthetic exhalation phase lung field image. The first parameter response map is determined according to the inhalation phase lung field image and the corresponding synthetic exhalation phase lung field image and a plurality of set threshold intervals.

19. The determination method of claim 9, wherein, Before determining the first parameter response map according to the inhalation phase image, the corresponding synthetic exhalation phase image and a plurality of set threshold intervals, the lung field segmentation model is used for lung field segmentation of the inhalation phase image and the synthetic exhalation phase image, to obtain a corresponding inhalation phase lung field image and a corresponding synthetic exhalation phase lung field image. The first parameter response map is determined according to the inhalation phase lung field image and the corresponding synthetic exhalation phase lung field image and a plurality of set threshold intervals.

20. The determination method of claim 14, wherein, Before determining the first parameter response map according to the inhalation phase image, the corresponding synthetic exhalation phase image and a plurality of set threshold intervals, the lung field segmentation model is used for lung field segmentation of the inhalation phase image and the synthetic exhalation phase image, to obtain a corresponding inhalation phase lung field image and a corresponding synthetic exhalation phase lung field image. The first parameter response map is determined according to the inhalation phase lung field image and the corresponding synthetic exhalation phase lung field image and a plurality of set threshold intervals.

21. A method of COPD phenotype determination, the method comprising: It comprises: An exhalation phase image and a plurality of set threshold intervals are obtained. The airway segmentation model is used for airway segmentation of the exhalation phase image to obtain a corresponding exhalation phase airway image. The exhalation phase image is synthesized into a corresponding synthetic inhalation phase image by using the preset synthesizer; wherein the exhalation phase image is synthesized into a corresponding synthetic inhalation phase image by using the preset synthesizer, which comprises training the preset synthesizer by using the exhalation phase image and its corresponding inhalation phase image for training; the exhalation phase image is synthesized into a corresponding synthetic inhalation phase image based on the trained preset synthesizer; A second parameter response map is determined according to the exhalation phase image and the corresponding synthetic inhalation phase image and a plurality of set threshold intervals. The second parameter response map is corrected by using the exhalation phase airway image, and then the COPD phenotype is determined.

22. The determination method according to claim 21, characterized in that, The second parameter response map is determined according to the exhalation phase image and the corresponding synthetic inhalation phase image and a plurality of set threshold intervals, which comprises: A registration operation is performed on the exhalation phase image and the corresponding synthetic inhalation phase image to obtain a corresponding second registration image; Determine a second parameter response map based on the second registered image and the plurality of preset threshold intervals.

23. The determination method according to any one of claims 21 or 22, characterized in that, The training process of the preset synthesizer using the expiratory phase image for training and its corresponding inspiratory phase image, comprises: using a second synthesizer G of the preset synthesizers E synthesizing the exhalation phase images for training into corresponding first synthesized inhalation phase images; using a first synthesizer G of the preset synthesizers I converting the first synthesized inhalation phase image into a first synthesized exhalation phase image; calculating a cycle consistency loss between the expiratory phase image for training and the first synthesized expiratory phase image; respectively performing convolution processing on the expiratory phase image for training and the first synthesized expiratory phase image to obtain corresponding expiratory phase feature maps and first synthesized expiratory phase feature maps; calculating a perception loss between the expiratory phase feature maps and the first synthesized expiratory phase feature maps; using a second preset discriminator D E , determining whether the first synthesized expiratory phase image is a real image or a synthesized image; and based on a result of the second preset discriminator D E computes an adversarial loss; Based on the cycle consistency loss, the perception loss and the adversarial loss, adjust network parameters of a first synthesizer G in the preset synthesizer I and a second synthesizer G E , to complete training of the preset synthesizer.

24. The determination method according to any one of claims 21 or 22, characterized in that, The process of synthesizing the expiratory phase image into a corresponding synthesized inspiratory phase image based on the trained preset synthesizer, comprises: obtaining a second synthesizer G in the preset synthesizer after training E ; based on the trained second synthesizer G of the preset synthesizers E performing convolution processing on the expiratory phase image to synthesize the expiratory phase image into a corresponding synthesized inspiratory phase image.

25. The determination method according to claim 23, characterized in that, The process of synthesizing the expiratory phase image into a corresponding synthesized inspiratory phase image based on the trained preset synthesizer, comprises: obtaining a second synthesizer G in the preset synthesizer after training E ; based on the trained second synthesizer G of the preset synthesizers E The expiration phase image is convoluted to synthesize a corresponding inhaled phase image.

26. The determination method according to any one of claims 21 or 22 or 25, characterized in that, Further comprising: segmenting lung blood vessels in the expiratory phase image using a preset blood vessel segmentation model to obtain a corresponding expiratory phase lung blood vessel image; Before performing the preset airway exclusion correction on the second parameter response map using the expiratory phase airway image, performing lung blood vessel exclusion correction on the second parameter response map to obtain a fifth corrected parameter response map; and performing preset airway exclusion correction on the parameter response map to obtain a sixth corrected parameter response map; and determining the COPD phenotype based on the sixth corrected parameter response map.

27. The determination method of claim 23, wherein, Further comprising: segmenting lung blood vessels in the expiratory phase image using a preset blood vessel segmentation model to obtain a corresponding expiratory phase lung blood vessel image; Before performing the preset airway exclusion correction on the second parameter response map using the expiratory phase airway image, performing lung blood vessel exclusion correction on the second parameter response map to obtain a fifth corrected parameter response map; and performing preset airway exclusion correction on the parameter response map to obtain a sixth corrected parameter response map; and determining the COPD phenotype based on the sixth corrected parameter response map.

28. The determination method of claim 24, wherein, Further comprising: segmenting lung blood vessels in the expiratory phase image using a preset blood vessel segmentation model to obtain a corresponding expiratory phase lung blood vessel image; Before performing the preset airway exclusion correction on the second parameter response map using the expiratory phase airway image, performing lung blood vessel exclusion correction on the second parameter response map to obtain a fifth corrected parameter response map; and performing preset airway exclusion correction on the parameter response map to obtain a sixth corrected parameter response map; and determining the COPD phenotype based on the sixth corrected parameter response map.

29. The method of determining according to any one of claims 21 or 22 or 25 or 27 or 28, characterized in that, In the preset airway exclusion correction, determining the preset airway comprises: determining the wall diameter of the airway affecting COPD; and determining the preset airway based on the wall diameter.

30. The determination method of claim 23, wherein, In the preset airway exclusion correction, determining the preset airway comprises: determining the wall diameter of the airway affecting COPD; and determining the preset airway based on the wall diameter.

31. The determination method of claim 24, wherein, In the preset airway exclusion correction, determining the preset airway comprises: determining the wall diameter of the airway affecting COPD; and determining the preset airway based on the wall diameter.

32. The determination method of claim 26, wherein, In the setting airway pruning correction, the setting airway is determined, including: determining the tube wall diameter corresponding to the airway affected by COPD; determining the setting airway based on the tube wall diameter.

33. The determination method of claim 29, wherein, The determination of the setting airway based on the tube wall diameter includes: respectively measuring the airway tube wall diameter in the expiratory phase airway image; determining the airway corresponding to the airway tube wall diameter greater than or equal to the tube wall diameter as the setting airway.

34. The method of determining according to any one of claims 30-32, wherein, The determination of the setting airway based on the tube wall diameter includes: respectively measuring the airway tube wall diameter in the expiratory phase airway image; determining the airway corresponding to the airway tube wall diameter greater than or equal to the tube wall diameter as the setting airway.

35. The determination method according to any one of claims 21 or 22 or 25 or 27 or 28, 30-33, wherein, Before determining the second parameter response map according to the expiratory phase image and the corresponding synthetic inspiratory phase image and a plurality of setting threshold intervals, lung field segmentation is performed on the expiratory phase image and the synthetic inspiratory phase image by using a lung field segmentation model to obtain a corresponding expiratory phase lung field image and a corresponding synthetic inspiratory phase lung field image. The second parameter response map is determined according to the expiratory phase lung field image and the corresponding synthetic inspiratory phase lung field image and a plurality of setting threshold intervals.

36. The determination method of claim 23, wherein, Before determining the second parameter response map according to the expiratory phase image and the corresponding synthetic inspiratory phase image and a plurality of setting threshold intervals, lung field segmentation is performed on the expiratory phase image and the synthetic inspiratory phase image by using a lung field segmentation model to obtain a corresponding expiratory phase lung field image and a corresponding synthetic inspiratory phase lung field image. The second parameter response map is determined according to the expiratory phase lung field image and the corresponding synthetic inspiratory phase lung field image and a plurality of setting threshold intervals.

37. The determination method of claim 24, wherein, Before determining the second parameter response map according to the expiratory phase image and the corresponding synthetic inspiratory phase image and a plurality of setting threshold intervals, lung field segmentation is performed on the expiratory phase image and the synthetic inspiratory phase image by using a lung field segmentation model to obtain a corresponding expiratory phase lung field image and a corresponding synthetic inspiratory phase lung field image. The second parameter response map is determined according to the expiratory phase lung field image and the corresponding synthetic inspiratory phase lung field image and a plurality of setting threshold intervals.

38. The determination method of claim 26, wherein, Before determining the second parameter response map according to the expiratory phase image and the corresponding synthetic inspiratory phase image and a plurality of setting threshold intervals, lung field segmentation is performed on the expiratory phase image and the synthetic inspiratory phase image by using a lung field segmentation model to obtain a corresponding expiratory phase lung field image and a corresponding synthetic inspiratory phase lung field image. The second parameter response map is determined according to the expiratory phase lung field image and the corresponding synthetic inspiratory phase lung field image and a plurality of setting threshold intervals.

39. The determination method of claim 29, wherein, Before determining the second parameter response map according to the expiratory phase image and the corresponding synthetic inspiratory phase image and a plurality of setting threshold intervals, lung field segmentation is performed on the expiratory phase image and the synthetic inspiratory phase image by using a lung field segmentation model to obtain a corresponding expiratory phase lung field image and a corresponding synthetic inspiratory phase lung field image. The second parameter response map is determined according to the expiratory phase lung field image and the corresponding synthetic inspiratory phase lung field image and a plurality of setting threshold intervals.

40. The determination method of claim 34, wherein, Before determining the second parameter response map according to the expiration phase image, the corresponding synthetic inspiration phase image, and a plurality of set threshold intervals, a lung field segmentation model is used to perform lung field segmentation on the expiration phase image and the synthetic inspiration phase image to obtain a corresponding expiration phase lung field image and a corresponding synthetic inspiration phase lung field image. The second parameter response map is determined according to the expiration phase lung field image, the corresponding synthetic inspiration phase lung field image, and a plurality of set threshold intervals.

41. A COPD phenotype prediction apparatus, comprising: The method comprises: An acquisition unit is configured to acquire an inspiration phase image and a plurality of set threshold intervals. A segmentation unit is configured to perform airway segmentation on the inspiration phase image by using a preset airway segmentation model to obtain a corresponding inspiration phase airway image. A synthesis unit is configured to synthesize the inspiration phase image into a corresponding synthetic expiration phase image by using a preset synthesizer. The method comprises: A determination unit is configured to determine a first parameter response map according to the inspiration phase image, the corresponding synthetic expiration phase image, and a plurality of set threshold intervals.

42. A COPD phenotype prediction apparatus, comprising: A modification unit is configured to perform set airway elimination correction on the first parameter response map by using the inspiration phase airway image to determine a COPD phenotype. The method comprises: An acquisition unit is configured to acquire an expiration phase image and a plurality of set threshold intervals. A segmentation unit is configured to perform airway segmentation on the expiration phase image by using a preset airway segmentation model to obtain a corresponding expiration phase airway image. A synthesis unit is configured to synthesize the expiration phase image into a corresponding synthetic inspiration phase image by using a preset synthesizer. The method comprises:

43. An electronic device, comprising: A determination unit is configured to determine a second parameter response map according to the expiration phase image, the corresponding synthetic inspiration phase image, and a plurality of set threshold intervals. A modification unit is configured to perform set airway elimination correction on the second parameter response map by using the expiration phase airway image to determine a COPD phenotype. The method comprises: A processor; 44. A computer-readable storage medium having stored thereon computer program instructions, wherein, A memory for storing processor-executable instructions; The processor is configured to invoke the instructions stored in the memory to perform the COPD phenotype determination method in any one of claims 1 to 40. The computer program instructions are executed by the processor to implement the COPD phenotype determination method in any one of claims 1 to 40.

Images