A fully automatic identification and assessment method for pulmonary ventilation dysfunction using dynamic chest radiography

Abstract

The present invention discloses a method for fully automatic identification and evaluation of dynamic chest radiography pulmonary ventilation dysfunction, the method comprising: obtaining a first original image and a first pulmonary ventilation imaging image of dynamic chest radiography in a DICOM file; obtaining a first lung field mask according to a semantic segmentation model and the first original image; performing segmentation processing on the first pulmonary ventilation imaging image according to the first original image and the first lung field mask to obtain a second pulmonary ventilation imaging image; training and testing an image classification model according to the first original image, the first lung field mask and the second pulmonary ventilation imaging image to obtain a target fully automatic identification and evaluation model. The present invention can improve the efficiency of analyzing images of dynamic chest radiography and can be widely used in the field of image processing technology.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a method for fully automatic recognition and evaluation of pulmonary ventilation dysfunction in dynamic chest photography. Background Art

[0002] At present, the post-processing workstation of dynamic chest radiography (DCR) shows that the pseudo-color lung field is larger than the lung field area shown in conventional chest X-rays and lacks objective quantitative data. The evaluation of lung ventilation function through DCR is highly dependent on the physician's understanding of lung respiratory movement and rich film reading experience, resulting in a lack of objective judgment in the evaluation results and low efficiency in analyzing dynamic chest radiography images. Summary of the invention

[0003] In view of this, the main purpose of the embodiments of the present invention is to provide a method for fully automatic identification and assessment of pulmonary ventilation dysfunction in dynamic chest radiography, in order to solve at least one problem in the prior art and to improve the efficiency of analyzing dynamic chest radiography images.

[0004] To achieve the above object, an embodiment of the present invention provides a method for fully automatic identification and assessment of pulmonary ventilation dysfunction in dynamic chest radiography, the method comprising the following steps:

[0005] Acquire the first original image of dynamic chest photography and the first lung ventilation imaging image in the DICOM file;

[0006] Acquire a first lung field mask according to the semantic segmentation model and the first original image;

[0007] According to the first original image and the first lung field mask, segmenting the first lung ventilation imaging image to obtain a second lung ventilation imaging image;

[0008] The image classification model is trained and tested according to the first original image, the first lung field mask and the second lung ventilation imaging image to obtain a fully automatic target recognition and evaluation model.

[0009] In some embodiments, before the step of acquiring the first lung field mask according to the semantic segmentation model and the first original image, the following steps are also included:

[0010] Acquire first original pixel data of the first original image;

[0011] It is determined whether the attribute of the DICOM file is a negative film, and when the attribute of the DICOM file is a negative film, the first original pixel data is inverted.

[0012] In some embodiments, obtaining a first lung field mask according to a semantic segmentation model and the first original image comprises the following steps:

[0013] Preprocessing the first original image to obtain a second original image;

[0014] Convert the second original pixel data of the second original image into a tensor and perform global normalization processing to obtain a third original image;

[0015] The third original image is input into the semantic segmentation model to generate the first lung field mask.

[0016] In some embodiments, preprocessing the first original image to obtain the second original image includes the following steps:

[0017] Acquire first original pixel data of the first original image;

[0018] Obtaining a mean value of the first original pixel data and a standard deviation of the first original pixel data;

[0019] According to the mean and the standard deviation, a preset range is set;

[0020] According to the preset range, the first original pixel data is cut to obtain third original pixel data;

[0021] Normalizing the third original pixel data to obtain a fourth original image;

[0022] Performing enhancement processing on the fourth original image and converting it into a grayscale image to obtain a fifth original image;

[0023] The pixel size of the fifth original image is adjusted to obtain the second original image.

[0024] In some embodiments, the segmentation process of the first lung ventilation imaging image according to the first original image and the first lung field mask to obtain the second lung ventilation imaging image includes the following steps:

[0025] Preprocessing and inverting the first lung field mask to obtain a second lung field mask;

[0026] Based on a matrix algorithm, according to the first original image and the second lung field mask, the lung field region of the first original image is removed to obtain a third original image;

[0027] performing normalization and binarization processing on the first pulmonary ventilation imaging image to obtain a third pulmonary ventilation imaging image;

[0028] multiplying the second lung field mask by the third lung ventilation imaging image to obtain an intermediate processed image;

[0029] The intermediate processed image is combined with the third original image to obtain the second lung ventilation imaging image.

[0030] In some embodiments, preprocessing and inverting the first lung field mask to obtain a second lung field mask comprises the following steps:

[0031] Traversing each element in the first lung field mask, adjusting the size of the first lung field mask, and obtaining a third lung field mask;

[0032] Normalizing the third lung field mask and performing data type conversion to obtain a fourth lung field mask;

[0033] Performing a hole filling operation on the fourth lung field mask to obtain a fifth lung field mask;

[0034] Acquire the area of ​​the connected domain of the fifth lung field mask;

[0035] According to the area, obtaining the largest connected domain to obtain a sixth lung field mask;

[0036] The sixth lung field mask is inverted to obtain the second lung field mask.

[0037] In some embodiments, the normalizing and binarizing the first pulmonary ventilation imaging image to obtain a third pulmonary ventilation imaging image comprises the following steps:

[0038] performing normalization processing on the first pulmonary ventilation imaging image to obtain a fourth pulmonary ventilation imaging image;

[0039] Setting a first threshold;

[0040] Setting pixel values ​​in the fourth lung ventilation imaging image that are greater than or equal to the first threshold to 1;

[0041] The pixel values ​​in the fourth lung ventilation imaging image that are less than the first threshold are set to 0 to obtain the third lung ventilation imaging image.

[0042] In some embodiments, the training and testing of the image classification model based on the first original image, the first lung field mask, and the second lung ventilation imaging image to obtain a fully automatic target recognition and evaluation model includes the following steps:

[0043] Dividing a target training set and a target test set according to the first original image, the first lung field mask, and the second lung ventilation imaging image;

[0044] Inputting the target training set into the image classification model for training;

[0045] The target test set is input into the trained image classification model for testing to obtain the target fully automatic recognition and evaluation model.

[0046] In some embodiments, dividing the target training set and the target test set according to the first original image, the first lung field mask, and the second lung ventilation imaging image comprises the following steps:

[0047] Preprocessing the first original image, the first lung field mask, and the second lung ventilation imaging image;

[0048] Stacking the preprocessed first original image, the preprocessed first lung field mask, and the preprocessed second lung ventilation imaging image to obtain a first tensor;

[0049] Adjusting the order of dimensions of the first tensor to obtain intermediate image data and corresponding labels;

[0050] Dividing an initial training set and a target test set according to the intermediate image data and the corresponding labels;

[0051] Performing data enhancement on the initial training set to obtain the target training set.

[0052] To achieve the above-mentioned purpose, another aspect of the embodiment of the present invention provides a dynamic chest radiography pulmonary ventilation dysfunction automatic identification and assessment system, the system comprising:

[0053] The first module is used to obtain a first original image of dynamic chest photography and a first lung ventilation imaging image in a DICOM file;

[0054] A second module is used to obtain a first lung field mask according to the semantic segmentation model and the first original image;

[0055] A third module is used to segment the first pulmonary ventilation imaging image according to the first original image and the first lung field mask to obtain a second pulmonary ventilation imaging image;

[0056] The fourth module is used to train and test the image classification model according to the first original image, the first lung field mask and the second lung ventilation imaging image to obtain a fully automatic target recognition and evaluation model.

[0057] To achieve the above-mentioned purpose, another aspect of an embodiment of the present invention provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the above-mentioned method for fully automatic identification and assessment of pulmonary ventilation dysfunction by dynamic chest photography.

[0058] To achieve the above-mentioned purpose, another aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned method for fully automatic identification and assessment of pulmonary ventilation dysfunction in dynamic chest photography.

[0059] To achieve the above-mentioned purpose, another aspect of the embodiment of the present invention provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the aforementioned method for fully automatic identification and assessment of pulmonary ventilation dysfunction in dynamic chest radiography.

[0060] The embodiments of the present invention include at least the following beneficial effects: The present invention provides a method for fully automatic identification and evaluation of pulmonary ventilation dysfunction in dynamic chest photography, which obtains a first original image and a first pulmonary ventilation imaging image of dynamic chest photography in a DICOM file; obtains a first lung field mask according to a semantic segmentation model and the first original image; performs segmentation processing on the first pulmonary ventilation imaging image according to the first original image and the first lung field mask to obtain a second pulmonary ventilation imaging image; trains and tests an image classification model according to the first original image, the first lung field mask and the second pulmonary ventilation imaging image to obtain a fully automatic target identification and evaluation model. Through the fully automatic target identification and evaluation model of the present invention, the original image and post-processed image of dynamic chest photography can be fully automatically analyzed and evaluated using a deep learning method, thereby improving the efficiency of analyzing images of dynamic chest photography. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0062] Figure 1This is a flow chart of a method for fully automatic identification and assessment of pulmonary ventilation dysfunction in dynamic chest photography provided by an embodiment of the present invention;

[0063] Figure 2 It is a schematic diagram of the process of lung field segmentation of PL image provided by an embodiment of the present invention;

[0064] Figure 3 Schematic diagram of the DCR image classification and evaluation process provided by an embodiment of the present invention;

[0065] Figure 4 is a receiver operating characteristic curve for identifying pulmonary ventilation dysfunction through DCR images provided by an embodiment of the present invention;

[0066] Figure 5 is a receiver operating characteristic curve for evaluating the severity of airflow limitation on DCR images of patients with ventilatory dysfunction provided by an embodiment of the present invention;

[0067] Figure 6 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention, and they are only examples of systems and methods consistent with some aspects of the embodiments of the present invention as detailed in the attached claims.

[0069] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the flow chart. The terms "first / S100" and "second / S200" in the specification and claims and the above-mentioned drawings may be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiment of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination".

[0070] The terms "at least one", "multiple", "each", "any", etc. used in the present invention, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0072] Before describing the embodiments of the present invention in detail, some nouns and terms involved in the embodiments of the present invention are first described. The nouns and terms involved in the embodiments of the present invention are subject to the following explanations.

[0073] Dynamic chest radiography (DCR) can perform real-time high-temporal resolution continuous photography with a radiation dose much lower than CT.

[0074] Pulmonary Low-frequency (PL) mode is an image post-processing mode that indirectly detects changes in ventilation by detecting changes in image pixel values ​​and other related signal features, and then visualizes the process of gas exchange in the lungs in the form of pseudo-color.

[0075] DICOM (Digital Imaging and Communications in Medicine) is an international standard (ISO 12052) for medical digital imaging and communications, which is widely used in the medical field, such as X-ray, CT, MRI, ultrasound, etc.

[0076] The Receiver Operating Characteristic (ROC) curve is a tool used to evaluate the performance of a binary classification model. It is represented by plotting the relationship between the true positive rate and the false positive rate under different threshold settings. The true positive rate is also called sensitivity or recall, and the false positive rate refers to the proportion of negative examples incorrectly classified as positive examples. The area under the ROC curve (AUC) can be used to quantify the overall performance of the model.

[0077] AUC (Area Under the Curve) is the area under the ROC curve. It is a value between 0 and 1 and is used to quantify the classification performance of the model. The closer the AUC is to 1, the better the classification performance of the model. An AUC of 0.5 means that the model has no distinguishing ability, which is equivalent to random guessing.

[0078] For the current DCR post-processing workstation, the lung field displayed in pseudo-color is larger than that displayed on conventional chest X-rays and lacks objective quantitative data. Therefore, physicians lack objectivity in their evaluation of DCR and spend a long time reading the films, and the efficiency of analyzing dynamic chest radiography images is low.

[0079] In view of this, if Figure 1 As shown, the embodiment of the present invention proposes a method for fully automatic identification and assessment of pulmonary ventilation dysfunction by dynamic chest photography, which may include but is not limited to steps S100 to S400:

[0080] Step S100, obtaining a first original image of dynamic chest photography and a first lung ventilation imaging image in a DICOM file;

[0081] Step S200, obtaining a first lung field mask according to the semantic segmentation model and the first original image;

[0082] Step S300, segmenting the first pulmonary ventilation imaging image according to the first original image and the first lung field mask to obtain a second pulmonary ventilation imaging image;

[0083] Step S400: training and testing an image classification model based on the first original image, the first lung field mask, and the second lung ventilation imaging image to obtain a fully automatic target recognition and evaluation model.

[0084] In some embodiments, the original grayscale image of the dynamic chest radiography (DCR) and the lung ventilation imaging (PL) image of the same frame at the end of deep inspiration of each patient can be obtained from the picture archiving and communication system (PACS) and saved as a DICOM file. The original image of the DCR is subjected to image post-processing operation in the PL module of the post-processing workstation to obtain the corresponding PL image.

[0085] In step S100 of some embodiments, the first original image and the first pulmonary ventilation imaging image of the DCR are read from the saved DICOM file. Exemplarily, the DICOM file is read using the dcmread function and pixel_array created in the dcm module to obtain the first original image and the corresponding first pulmonary ventilation imaging image.

[0086] Before step S200 in some embodiments, the method further includes: obtaining the first original pixel data of the first original image; determining whether the attribute of the DICOM file is a negative film, and when the attribute of the DICOM file is a negative film, inverting the first original pixel data. Exemplarily, the pixel data of the original image of the DCR in the DICOM file can be read through the dcmread function and pixel_array created in the dcm module; then, the pixel data is processed according to the value of the PresentationLUTShape attribute, and if the PresentationLUTShape attribute in the DICOM file is a negative film, the pixel data is inverted, that is, all pixel values ​​of the image are multiplied by -1, and the maximum absolute value in the original pixel data is added to each pixel value.

[0087] In some embodiments, step S200 may include but is not limited to steps S210 to S230:

[0088] Step S210, preprocessing the first original image to obtain a second original image;

[0089] Step S220, converting the second original pixel data of the second original image into a tensor and performing global normalization processing to obtain a third original image;

[0090] Step S230: input the third original image into the semantic segmentation model to generate the first lung field mask.

[0091] In some embodiments, step S210 may include but is not limited to steps S211 to S217:

[0092] Step S211, obtaining first original pixel data of the first original image;

[0093] Step S212, obtaining a mean value of the first original pixel data and a standard deviation of the first original pixel data;

[0094] Step S213, setting a preset range according to the mean and the standard deviation;

[0095] Step S214, cutting the first original pixel data according to the preset range to obtain third original pixel data;

[0096] Step S215, normalizing the third original pixel data to obtain a fourth original image;

[0097] Step S216, performing enhancement processing on the fourth original image and converting it into a grayscale image to obtain a fifth original image;

[0098] Step S217: adjusting the pixel size of the fifth original image to obtain the second original image.

[0099] In some embodiments, in steps S211 to S217, the original image of the DCR is preprocessed. For example, the mean value of the original image pixel data of the current DCR is calculated. and standard deviation ; will be with the mean Two standard deviations The range is set to a preset range, then the preset range can be ; The pixel data of the original image is sheared to ensure that the pixel data is within the preset range; then the pixel data obtained after shearing is normalized according to the maximum and minimum value normalization formula, and the pixel data of the original image of the sheared DCR is scaled to the range of 0-225; then the normalized original image of the DCR can be enhanced by an adaptive histogram equalization method, and the enhanced image is converted into a grayscale image and resized to 256×256 pixels; wherein the maximum and minimum value normalization formula is:

[0100] ;

[0101] In the formula, It is the normalized pixel data, ranging from 0 to 225; is the minimum value in the pixel data; is the maximum value in the pixel data; is the pixel data in the pixel data set.

[0102] In some embodiments, in steps S220 to S230, the preprocessed DCR original image pixel data may be converted into a PyTorch tensor and globally normalized. The pretrained FCDenseNet103 model is loaded, and the lung field of the DCR original image may be segmented through the pretrained FCDenseNet103 model, that is, the DCR original image data that has been converted into a PyTorch tensor and globally normalized is input into the pretrained FCDenseNet103 model, and the image is segmented into the lung field to generate a lung field mask.

[0103] In some embodiments, the semantic segmentation model can be a pre-trained FCDenseNet103 model, which uses a DenseNet structure and includes transition layers (Transition Down and Transition Up) to achieve size changes in feature maps. The following is the detailed architecture of the model:

[0104] (1) Define a dense layer, including batch normalization, ReLU activation function, convolution operation and two-dimensional Dropout, to extract features and enhance the nonlinear representation and generalization capabilities of the model. Among them, batch normalization of the input helps to accelerate convergence and stabilize training; use the ReLU activation function to increase nonlinear characteristics; apply a 3x3 convolution kernel, define the number of output channels as growth_rate, and use padding of 1 to keep the input and output sizes consistent; use the two-dimensional Dropout method to set the input unit to zero with a probability of 0.2 to prevent overfitting.

[0105] (2) Define a dense block, which contains multiple DenseLayers. If upsample is True, feature concatenation is performed after the output of each DenseLayer, and the output of each DenseLayer is stored in the new_features array; if upsample is False, feature concatenation is performed in order as usual.

[0106] (3) Define the downsampling layer (Transition Down). The number of DenseBlocks contained in the downsampling path is (4, 5, 7, 10, 12) respectively. The size of the feature map is halved through batch normalization, ReLU activation, convolution, 2D Dropout and maximum pooling.

[0107] (4) Define the upsampling layer (Transition Up). The number of DenseBlocks contained in the upsampling path is (12, 10, 7, 5, 4) respectively. During the decoding process, the feature map size is doubled through the transposed convolution layer and connected with the feature map in the encoding stage to restore the high-resolution feature map.

[0108] (5) Define the Bottleneck module (Bottleneck) and use DenseBlock to implement the Bottleneck structure, which is used to connect the upsampling and downsampling paths in U-Net or similar architectures.

[0109] (6) Define the cropping function to crop the feature map layer to the specified height and width.

[0110] In some embodiments, step S300 may include but is not limited to steps S310 to S350:

[0111] Step S310, preprocessing and inverting the first lung field mask to obtain a second lung field mask;

[0112] Step S320, based on a matrix algorithm, according to the first original image and the second lung field mask, removing the lung field area of ​​the first original image to obtain a third original image;

[0113] Step S330, performing normalization and binarization processing on the first pulmonary ventilation imaging image to obtain a third pulmonary ventilation imaging image;

[0114] Step S340, multiplying the second lung field mask by the third lung ventilation imaging image to obtain an intermediate processed image;

[0115] Step S350: combining the intermediate processed image with the third original image to obtain the second lung ventilation imaging image.

[0116] In some embodiments, step S310 may include but is not limited to steps S311 to S316:

[0117] Step S311, traversing each element in the first lung field mask, adjusting the size of the first lung field mask, and obtaining a third lung field mask;

[0118] Step S312, normalizing the third lung field mask and performing data type conversion to obtain a fourth lung field mask;

[0119] Step S313, performing a hole filling operation on the fourth lung field mask to obtain a fifth lung field mask;

[0120] Step S314, obtaining the area of ​​the connected domain of the fifth lung field mask;

[0121] Step S315, obtaining the largest connected domain according to the area, and obtaining a sixth lung field mask;

[0122] Step S316: invert the sixth lung field mask to obtain the second lung field mask.

[0123] In some embodiments, in steps S311 to S312, the first lung field mask generated by segmentation is loaded, each element in each first lung field mask is traversed, and the first lung field mask is adjusted to the same size as the PL image using the resize function of OpenCV, and the data of the adjusted third lung field mask is normalized to between 0 and 1, and converted to np.uint8 type so that its pixel value is in the range of [0,255] to obtain the fourth lung field mask. The second threshold can be set to 0.7, and the pixel values ​​in the lung field mask that are greater than or equal to the second threshold are set to 255, and the pixel values ​​in the lung field mask that are less than the second threshold are set to 0.

[0124] In step S313 of some embodiments, the binary_fill_holes function of SciPy may be used to fill the holes in the fourth lung field mask to ensure that all regions in the fourth lung field mask are connected, thereby obtaining the fifth lung field mask.

[0125] In step S314 of some embodiments, the created function keep_largest_connected_components is called to obtain the area information of the connected domain. Since the background (label 0) is not part of the connected area, the background (label 0) is excluded, that is, the background is not included in the area list.

[0126] In steps S315 to S316 of some embodiments, the obtained connected domains are sorted according to their areas, and the labels of the first two regions can be obtained from the sorted list, that is, the labels of the two connected domains with the largest areas are found, and these two connected domains are retained as the largest connected domains in the lung field mask, ensuring that the processed lung field mask is the segmented lung field area, and obtaining the sixth lung field mask; then the sixth lung field mask is inverted to obtain the second lung field mask.

[0127] In step S320 of some embodiments, the first original image and the second lung field mask of the DCR are calculated by a matrix algorithm, thereby removing the double lung field regions of the first original image and retaining only the regions other than the lung fields, and then a third original image can be obtained. Then, the following formula is obtained:

[0128] ;

[0129] In the formula, represents the third original image, represents the second lung field mask, is the element-wise multiplication (Hadamard product).

[0130] In some embodiments, step S330 may include but is not limited to steps S331 to S334:

[0131] Step S331, normalizing the first pulmonary ventilation imaging image to obtain a fourth pulmonary ventilation imaging image;

[0132] Step S332, setting a first threshold;

[0133] Step S333, setting the pixel values ​​in the fourth lung ventilation imaging image that are greater than or equal to the first threshold to 1;

[0134] Step S334: setting pixel values ​​in the fourth pulmonary ventilation imaging image that are less than the first threshold to 0 to obtain the third pulmonary ventilation imaging image.

[0135] In steps S331 to S334 of some embodiments, a first pulmonary ventilation imaging (PL) image in a DICOM file is loaded, the image data is normalized, and its pixel value range is scaled to [0, 1]. Exemplarily, the first pulmonary ventilation imaging image is normalized to obtain a fourth pulmonary ventilation imaging image; the first threshold is set to 0.9, and the fourth pulmonary ventilation imaging image is binarized, that is, the pixel values ​​in the fourth pulmonary ventilation imaging image that are greater than or equal to the first threshold are set to 1, and the pixel values ​​in the fourth pulmonary ventilation imaging image that are less than the first threshold are set to 0, and then a third pulmonary ventilation imaging image can be obtained.

[0136] In steps S340 to S350 of some embodiments, Figure 2 As shown, the second lung field mask is multiplied by the third pulmonary ventilation imaging image, thereby removing the first redundant area of ​​the third pulmonary ventilation imaging image that exceeds the lung field area (i.e., the redundant blue area of ​​the third pulmonary ventilation imaging image that exceeds the lung field area), and removing the area where the connected domain area is less than the third threshold, and then using a full 1 kernel of size 10×10 to perform an open operation, remove the background information other than the lung field in the third pulmonary ventilation imaging image, remove the second redundant area with a connected domain area less than 1200 (i.e., the redundant black area with a connected domain area less than 1200), and obtain the target blue area containing only the lung field after segmentation as the intermediate processing image 22, wherein the third threshold can be set to 500. The intermediate processing image 22 is combined with the third original image 21 with the lung field area removed to obtain the second pulmonary ventilation imaging image 23. Then there is the following formula:

[0137] ;

[0138] In the formula, represents the second lung ventilation imaging image, For the second lung field mask, This is the third lung ventilation imaging image. is the element-wise multiplication (Hadamard product).

[0139] In some embodiments, the consistency between the PLA of the intermediate processed image and the PLA of the third original image can be compared by secondary random sampling. The secondary random sampling can be used to determine whether the segmentation of the semantic segmentation model is correct. Wherein, PLA refers to the conventional lung field projection area shown on the anteroposterior chest X-ray image.

[0140] In some embodiments, step S400 may include but is not limited to steps S410 to S430:

[0141] Step S410, dividing a target training set and a target test set according to the first original image, the first lung field mask, and the second lung ventilation imaging image;

[0142] Step S420, inputting the target training set into the image classification model for training;

[0143] Step S430, inputting the target test set into the trained image classification model for testing to obtain the target fully automatic recognition and evaluation model.

[0144] In some embodiments, step S410 may include but is not limited to steps S411 to S415:

[0145] Step S411, preprocessing the first original image, the first lung field mask, and the second lung ventilation imaging image;

[0146] Step S412, stacking the preprocessed first original image, the preprocessed first lung field mask, and the preprocessed second lung ventilation imaging image to obtain a first tensor;

[0147] Step S413, adjusting the dimensional order of the first tensor to obtain intermediate image data and corresponding labels;

[0148] Step S414, dividing the initial training set and the target test set according to the intermediate image data and the corresponding labels;

[0149] Step S415, performing data enhancement on the initial training set to obtain the target training set.

[0150] In steps S411 to S414 of some embodiments, the first original image, the first lung field mask, and the second lung ventilation imaging image can be preprocessed by cropping, normalization, data enhancement, etc., and the preprocessed first original image, the first lung field mask, and the second lung ventilation imaging image can be stacked into a three-layer tensor according to the third dimension; the processed image data and the corresponding labels can be returned as a sample; in addition, the tensor array is transposed to adjust the dimensional order of the tensor, and a processed intermediate image data and its corresponding label for training and testing are returned; the initial training set and the target test set are divided using 10-fold cross validation to ensure that the proportion of abnormal and normal samples in each fold is the same as that in the overall data set.

[0151] In step S415 of some embodiments, data enhancement is performed on the intermediate image data in the initial training set to increase the diversity and generalization ability of the data. For example, the image size is adjusted to 224×224 pixels, the image is standardized to the input size required by the deep learning model, and the images are ensured to have the same size; the image is horizontally flipped, blurred, and vertically flipped with a probability of 50%; Gaussian noise, optical distortion, grid distortion, etc. are added to the image with a probability of 20%, and the brightness and contrast of the image are randomly adjusted with a probability of 20%.

[0152] In step S420 of some embodiments, Figure 3 As shown, the intermediate image data of the target training set after data augmentation is input into the pre-trained image classification model for training.

[0153] In some embodiments, the image classification model can be a pre-trained ResNet 18 model, the first layer of which is a 7x7 convolution kernel with a step size of 2 and an output channel number of 64, followed by a 2x2 maximum pooling layer; each of the subsequent four stages contains several convolution layers, and downsampling is performed at the end of each stage; these convolution layers use 3x3 convolution kernels with a step size of 1; each stage contains multiple residual blocks, each residual block contains two 3x3 convolution layers, and a connection across the residual (shortcut connection), and dimension matching is performed through identity mapping or 1x1 convolution to ensure that the input and output can be added; after the last residual block, the feature map size is aggregated to 1x1 through global average pooling; the last fully connected layer of the model is replaced by a linear layer with an output category number of 2.

[0154] In some embodiments, the Adam optimizer is used to update the parameters of the model. The learning rate scheduler is set to multiply the learning rate by 0.1 every 15 epochs. Train and validate each epoch. Enable dropout and batch normalization. During the training process, the following operations are performed on each batch: forward propagation calculates the output, uses the cross entropy loss function to calculate the loss, back propagates to update the parameters, calculates and accumulates the training loss and accuracy; after each epoch, the average loss and accuracy of the training set are calculated.

[0155] In step S430 of some embodiments, the trained model is tested using the target test set, the results of 10 runs are averaged, and the average AUC (area under the receiver operating characteristic curve), accuracy, sensitivity, specificity and precision are used as indicators to evaluate the performance of the model. Figure 3As shown, the image data is input into the obtained target automatic recognition and evaluation model, and the evaluation result of the image is output. The image passes through the initial convolution layer of the model to extract preliminary features. The feature map is input into the residual block for deep feature extraction. The output of the last residual block passes through the global average pooling layer to reduce the size of each feature map to 1x1. The pooled feature map is flattened to form a one-dimensional vector. The flattened vector is input into the fully connected layer for final feature mapping. The classification score (logits) is obtained through the fully connected layer, and the output is the score of each category. The logits are converted into a probability distribution through the Softmax layer. According to the probability output by the Softmax layer, the category with the highest probability is selected as the final classification result.

[0156] The target automatic recognition and evaluation model of the embodiment of the present invention performs well in the task of identifying pulmonary ventilation dysfunction, such as Figure 4 The average ROC curve of the target classification model for identifying pulmonary ventilation dysfunction is shown in Figure 1. The AUC value of the model is 0.836, indicating that the model has a strong ability to correctly identify pulmonary ventilation dysfunction under all possible classification thresholds. At the same time, the target automatic recognition and evaluation model of the embodiment of the present invention performs well in the task of evaluating the severity of airflow limitation, such as Figure 5 The average ROC curve of the target classification model for evaluating the severity of airflow limitation is shown in Figure 1. The AUC value of this model is 0.817, indicating that this model has a strong ability to correctly evaluate the severity of airflow limitation under all possible classification thresholds.

[0157] The embodiment of the present invention further provides a dynamic chest radiography pulmonary ventilation dysfunction fully automatic identification and evaluation system, which can implement the above-mentioned dynamic chest radiography pulmonary ventilation dysfunction fully automatic identification and evaluation method, the system comprising:

[0158] The first module is used to obtain a first original image of dynamic chest photography and a first lung ventilation imaging image in a DICOM file;

[0159] A second module is used to obtain a first lung field mask according to the semantic segmentation model and the first original image;

[0160] A third module is used to segment the first pulmonary ventilation imaging image according to the first original image and the first lung field mask to obtain a second pulmonary ventilation imaging image;

[0161] The fourth module is used to train and test the image classification model according to the first original image, the first lung field mask and the second lung ventilation imaging image to obtain a fully automatic target recognition and evaluation model.

[0162] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0163] The embodiment of the present invention further provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the above-mentioned method for fully automatic identification and assessment of pulmonary ventilation dysfunction in dynamic chest photography is implemented. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0164] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0165] refer to Figure 6 , Figure 6 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0166] The processor 601 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention;

[0167] The memory 602 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM). The memory 602 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 602, and the processor 601 calls and executes a dynamic chest radiography pulmonary ventilation dysfunction fully automatic identification and evaluation method of the embodiment of the present invention;

[0168] Input / output interface 603, used to implement information input and output;

[0169] Communication interface 604, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0170] A bus 605 that transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604);

[0171] The processor 601 , the memory 602 , the input / output interface 603 and the communication interface 604 are connected to each other in communication within the device via a bus 605 .

[0172] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for fully automatic identification and assessment of pulmonary ventilation dysfunction using dynamic chest radiography.

[0173] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0174] The embodiment of the present invention also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the aforementioned dynamic chest radiography pulmonary ventilation dysfunction fully automatic identification and assessment method.

[0175] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0176] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0177] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0178] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0179] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0180] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0181] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0182] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0183] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A method for fully automatic identification and assessment of pulmonary ventilation dysfunction using dynamic chest radiography, characterized in that: The following steps are involved: Acquire the first original image of dynamic chest photography and the first lung ventilation imaging image in the DICOM file; Acquire a first lung field mask according to the semantic segmentation model and the first original image; According to the first original image and the first lung field mask, segmenting the first lung ventilation imaging image to obtain a second lung ventilation imaging image; According to the first original image, the first lung field mask and the second lung ventilation imaging image, an image classification model is trained and tested to obtain a fully automatic target recognition and evaluation model; The step of segmenting the first pulmonary ventilation imaging image according to the first original image and the first lung field mask to obtain a second pulmonary ventilation imaging image comprises the following steps: Preprocessing and inverting the first lung field mask to obtain a second lung field mask; Based on a matrix algorithm, according to the first original image and the second lung field mask, the lung field region of the first original image is removed to obtain a third original image; performing normalization and binarization processing on the first pulmonary ventilation imaging image to obtain a third pulmonary ventilation imaging image; multiplying the second lung field mask by the third lung ventilation imaging image to obtain an intermediate processed image; The intermediate processed image is combined with the third original image to obtain the second lung ventilation imaging image.

2. The method for fully automatic identification and assessment of pulmonary ventilation dysfunction by dynamic chest radiography according to claim 1, characterized in that: Before the step of obtaining the first lung field mask according to the semantic segmentation model and the first original image, the following steps are also included: Acquire first original pixel data of the first original image; It is determined whether the attribute of the DICOM file is a negative film, and when the attribute of the DICOM file is a negative film, the first original pixel data is inverted.

3. The method for fully automatic identification and assessment of pulmonary ventilation dysfunction by dynamic chest radiography according to claim 1, characterized in that: The step of obtaining a first lung field mask according to the semantic segmentation model and the first original image comprises the following steps: Preprocessing the first original image to obtain a second original image; Convert the second original pixel data of the second original image into a tensor and perform global normalization processing to obtain a third original image; The third original image is input into the semantic segmentation model to generate the first lung field mask.

4. The method for fully automatic identification and assessment of pulmonary ventilation dysfunction by dynamic chest radiography according to claim 3, characterized in that: The preprocessing of the first original image to obtain a second original image comprises the following steps: Acquire first original pixel data of the first original image; Obtaining a mean value of the first original pixel data and a standard deviation of the first original pixel data; According to the mean and the standard deviation, a preset range is set; According to the preset range, the first original pixel data is cut to obtain third original pixel data; Normalizing the third original pixel data to obtain a fourth original image; Performing enhancement processing on the fourth original image and converting it into a grayscale image to obtain a fifth original image; The pixel size of the fifth original image is adjusted to obtain the second original image.

5. The method for fully automatic identification and assessment of pulmonary ventilation dysfunction by dynamic chest radiography according to claim 1, characterized in that: The preprocessing and inverting the first lung field mask to obtain a second lung field mask comprises the following steps: Traversing each element in the first lung field mask, adjusting the size of the first lung field mask, and obtaining a third lung field mask; Normalizing the third lung field mask and performing data type conversion to obtain a fourth lung field mask; Performing a hole filling operation on the fourth lung field mask to obtain a fifth lung field mask; Acquire the area of ​​the connected domain of the fifth lung field mask; According to the area, obtaining the largest connected domain to obtain a sixth lung field mask; The sixth lung field mask is inverted to obtain the second lung field mask.

6. The method for fully automatic identification and assessment of pulmonary ventilation dysfunction by dynamic chest radiography according to claim 1, characterized in that: The step of normalizing and binarizing the first pulmonary ventilation imaging image to obtain a third pulmonary ventilation imaging image comprises the following steps: performing normalization processing on the first pulmonary ventilation imaging image to obtain a fourth pulmonary ventilation imaging image; Setting a first threshold; Setting pixel values ​​in the fourth lung ventilation imaging image that are greater than or equal to the first threshold to 1; The pixel values ​​in the fourth lung ventilation imaging image that are less than the first threshold are set to 0 to obtain the third lung ventilation imaging image.

7. The method for fully automatic identification and assessment of pulmonary ventilation dysfunction using dynamic chest radiography according to claim 1, characterized in that: The method of training and testing the image classification model according to the first original image, the first lung field mask, and the second lung ventilation imaging image to obtain a fully automatic target recognition and evaluation model includes the following steps: Dividing a target training set and a target test set according to the first original image, the first lung field mask, and the second lung ventilation imaging image; Inputting the target training set into the image classification model for training; The target test set is input into the trained image classification model for testing to obtain the target fully automatic recognition and evaluation model.

8. The method for fully automatic identification and assessment of pulmonary ventilation dysfunction using dynamic chest radiography according to claim 7, characterized in that: The step of dividing a target training set and a target test set according to the first original image, the first lung field mask, and the second lung ventilation imaging image comprises the following steps: Preprocessing the first original image, the first lung field mask, and the second lung ventilation imaging image; Stacking the preprocessed first original image, the preprocessed first lung field mask, and the preprocessed second lung ventilation imaging image to obtain a first tensor; Adjusting the order of dimensions of the first tensor to obtain intermediate image data and corresponding labels; Dividing an initial training set and a target test set according to the intermediate image data and the corresponding labels; Performing data enhancement on the initial training set to obtain the target training set.

9. A dynamic chest radiography pulmonary ventilation dysfunction automatic identification and assessment system, characterized in that: include: The first module is used to obtain a first original image of dynamic chest photography and a first lung ventilation imaging image in a DICOM file; A second module is used to obtain a first lung field mask according to the semantic segmentation model and the first original image; A third module is used to segment the first pulmonary ventilation imaging image according to the first original image and the first lung field mask to obtain a second pulmonary ventilation imaging image; A fourth module is used to train and test the image classification model according to the first original image, the first lung field mask and the second lung ventilation imaging image to obtain a fully automatic target recognition and evaluation model; Wherein, the third module is specifically used for: Preprocessing and inverting the first lung field mask to obtain a second lung field mask; Based on a matrix algorithm, according to the first original image and the second lung field mask, the lung field region of the first original image is removed to obtain a third original image; performing normalization and binarization processing on the first pulmonary ventilation imaging image to obtain a third pulmonary ventilation imaging image; multiplying the second lung field mask by the third lung ventilation imaging image to obtain an intermediate processed image; The intermediate processed image is combined with the third original image to obtain the second lung ventilation imaging image.

Images