An automatic pulmonary embolism detection method based on CT pulmonary angiography images

Abstract

The present invention discloses an automatic pulmonary embolism detection method based on CT pulmonary angiography images, which relates to the technology of medical image processing and segmentation. Obtain dual-phase CT pulmonary angiography images; construct an embolism region segmentation model, input the dual-phase CT pulmonary angiography images into the embolism region segmentation model for embolism detection and analysis to obtain dual-phase pulmonary embolism detection images; construct a contrast image registration training model; input the dual-phase pulmonary embolism detection images into the contrast image registration training model for spatial transformation based on the displacement field to obtain a pulmonary artery embolism region detection image and a pulmonary vein embolism region detection image; judge whether the isolated pulmonary embolism region is a positive embolism region according to the overlap ratio relationship between the isolated pulmonary embolism regions included in the pulmonary artery embolism region detection image and the pulmonary vein embolism region detection image. The present invention combines the image information of two phases and can more effectively identify false positive embolism regions.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing and segmentation, and more specifically, to an automatic pulmonary embolism detection method based on CT pulmonary angiography images. Background Art

[0002] Pulmonary embolism is a syndrome mainly manifested by pulmonary circulation and respiratory failure, and is a common disease with a high risk of death. Early diagnosis and accurate risk stratification of pulmonary embolism are the difficulties in its diagnosis and treatment. At present, CT pulmonary angiography is the preferred examination method for diagnosing pulmonary embolism. The indicators such as the embolism site, embolism area, and right heart function indicated by CT pulmonary angiography are important bases for risk stratification of pulmonary embolism and selection of treatment strategies. However, due to the different understandings of pulmonary embolism disease among clinicians and radiologists, and the uneven abilities of reading CT pulmonary angiography films, it seriously affects the diagnosis, treatment, and prognosis of patients with pulmonary embolism. Therefore, an automatic pulmonary embolism detection method based on CT pulmonary angiography images plays a very important role.

[0003] Pulmonary embolism presents a sign of low-density filling defect in CT pulmonary angiography images. During the CT pulmonary angiography process, due to the accuracy of the physician's grasp of the scanning timing, the flow of contrast agent in the blood vessels, and the complexity of the patient's condition, low-density areas such as artifacts and non-visualization will appear in the arterial-phase angiography images, increasing the false positive rate of current automatic pulmonary embolism detection methods, thus leading to misdiagnosis. To reduce the misdiagnosis rate, a large number of studies have been carried out in this field.

[0004] Such as the prior art one: A pulmonary embolism detection system, medium and electronic device disclosed in the patent application No. CN202110110806.6. First, it obtains a CT image and performs preprocessing, and then inputs the preprocessed image into a specific neural network, which outputs two values, 0 or 1. 0 indicates no embolism lesion, and 1 indicates the presence of an embolism lesion, as Figure 1 shown.

[0005] This technical process is simple and easy to implement. However, this technology only judges whether there is an embolism lesion in a certain image, lacking relevant information about the lesion. Since it is based on single-phase images for recognition, it cannot effectively distinguish whether the low-density sign area in the image is a lesion.

[0006] Another example is the prior art two: A pulmonary embolism detection system based on a convolutional neural network disclosed in the patent application No. CN201910930769.6. A convolutional neural network for object detection and classification is constructed and trained, and the segmented CT pulmonary angiography images are input into the network. The network outputs the position, confidence level, and size information of the candidate embolism area, and then a false positive removal network is used to remove some false positive embolism candidate targets, as Figure 2 shown.

[0007] This technology can detect the embolism area in the image, providing richer diagnostic information. However, since it is based on single-phase images for recognition, it cannot effectively distinguish whether the area with low-density signs in the image is a lesion. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide an automatic pulmonary embolism detection method based on CT pulmonary angiography images to more effectively identify false-positive embolism areas in view of the deficiencies of the prior art.

[0009] An automatic pulmonary embolism detection method based on CT pulmonary angiography images according to the present invention includes the following steps:

[0010] Step 1: Obtain dual-phase CT pulmonary angiography images;

[0011] Step 2: Construct an embolism area segmentation model based on a convolutional neural network, input the dual-phase CT pulmonary angiography images into the embolism area segmentation model for embolism detection and analysis to obtain dual-phase pulmonary embolism detection images;

[0012] Step 3: Construct a contrast-enhanced image registration training model based on a convolutional neural network; input the dual-phase pulmonary embolism detection images into the contrast-enhanced image registration training model for spatial transformation based on the displacement field to obtain pulmonary artery embolism area detection images and pulmonary vein embolism area detection images;

[0013] Step 4: Determine whether the isolated pulmonary embolism area is a positive embolism area according to the overlap ratio relationship between the isolated pulmonary embolism areas included in the pulmonary artery embolism area detection images and the pulmonary vein embolism area detection images.

[0014] The embolism area segmentation model is composed of n + 1 residual modules;

[0015] The residual modules are divided into downsampling residual modules and upsampling residual modules;

[0016] The downsampling residual module includes a first convolutional layer, a second convolutional layer, and a third convolutional layer; the first convolutional layer, the second convolutional layer, and the third convolutional layer are sequentially connected through the activation function Relu, and after the third convolutional layer is added to the input data of the downsampling residual module through the activation function Relu, downsampling binary segmentation data is obtained;

[0017] The upsampling residual module includes a transposed convolution layer, a fourth convolution layer, and a fifth convolution layer; the transposed convolution layer, the fourth convolution layer, and the fifth convolution layer are sequentially connected through the activation function Relu. After the output of the fifth convolution layer is added to the input data of the upsampling residual module through the activation function Relu, upsampled binary segmentation data is obtained.

[0018] Where n is a natural number greater than or equal to 1.

[0019] Among the n + 1 residual modules, the first to the n / 2-th residual modules are downsampling residual modules; the (n / 2 + 1)-th to the (n + 1)-th residual modules are upsampling residual modules.

[0020] The contrast image registration training model includes

[0021] An encoding and decoding module, which is composed of m convolution modules, is used to input two standard three-dimensional pulse-phase images, perform downsampling encoding operations on the two standard three-dimensional pulse-phase images through the first to the m / 2-th convolution layer modules, and then perform upsampling decoding through the (m / 2 + 1)-th to the m-th convolution layer modules to output a displacement field image.

[0022] A spatial transformation module, which is used to perform spatial transformation on all pixels in the three-dimensional pulmonary vein phase image of the two standard three-dimensional pulse-phase images according to the displacement field image to obtain a registered venous phase image.

[0023] Where m is a natural number greater than or equal to 1.

[0024] The contrast image registration training model further includes an iterative training module, which is used to continue encoding, decoding, and spatial transformation training on the registered venous phase image and the three-dimensional pulmonary artery phase image in the two standard three-dimensional pulse-phase images through iterative training, so as to minimize the difference between the registered pulmonary vein phase image and the three-dimensional pulmonary artery phase image, and complete the registration training.

[0025] The two standard three-dimensional pulse-phase images are preprocessed three-dimensional pulmonary artery phase images and three-dimensional pulmonary vein phase images.

[0026] The preprocessing method is to truncate the CT value of the standard three-dimensional pulse-phase image and normalize it to a value between 0 and 1.

[0027] Inputting the dual-phase pulmonary embolism detection image into the contrast image registration training model for spatial transformation based on the displacement field specifically means

[0028] Taking the dual-phase pulmonary embolism detection image as the input image of the spatial transformation module, and performing spatial transformation on the dual-phase pulmonary embolism detection image based on the displacement field image obtained from the two standard three-dimensional pulse-phase images, so as to obtain a pulmonary artery embolism region detection image and a pulmonary vein embolism region detection image.

[0029] Identify the isolated pulmonary artery embolism region in the detected image of the pulmonary artery embolism region, and identify the isolated pulmonary vein embolism region in the detected image of the pulmonary vein embolism region;

[0030] Preset a standard overlap ratio; calculate the overlap ratio between the isolated pulmonary artery embolism region and the isolated pulmonary vein embolism region; if the overlap ratio is greater than or equal to the standard overlap ratio, determine that the embolism region is a positive embolism region; otherwise, determine it as a false positive pulmonary embolism region.

[0031] Advantageous effects

[0032] The advantages of the present invention are as follows: compared with the existing solutions, the present invention combines the image information of two phases, and uses a convolutional neural network to realize the identification, matching and confirmation of suspected embolism regions in dual-phase pulmonary vascular images. While detecting the embolism region information, it can effectively reduce the detection rate of false positive embolisms. Description of the drawings

[0033] Figure 1 Is a flowchart of pulmonary embolism detection for the prior art one;

[0034] Figure 2 Is a flowchart of pulmonary embolism detection for the prior art two;

[0035] Figure 3 Is a flowchart of the automatic pulmonary embolism detection method of the present invention;

[0036] Figure 4 Is a schematic diagram of the segmentation network of the embolism region segmentation model of the present invention;

[0037] Figure 5 Is a schematic diagram of the structure of the downsampling residual module of the present invention;

[0038] Figure 6 Is a schematic diagram of the structure of the upsampling residual module of the present invention;

[0039] Figure 7 Is a schematic diagram of the training structure of the contrast image registration training model of the present invention;

[0040] Figure 8 Is a schematic diagram of the registration process of the contrast image registration training model for the embolism detection image of the present invention;

[0041] Figure 9 Is a schematic diagram of the modular structure of the automatic pulmonary embolism detection system of the present invention. Detailed implementation manners

[0042] The present invention will be further described below in conjunction with embodiments, but it does not constitute any limitation to the present invention. Any limited number of modifications made by anyone within the scope of the claims of the present invention are still within the scope of the claims of the present invention.

[0043] Referring to Figure 3 , an automatic pulmonary embolism detection method based on CT pulmonary angiography images of the present invention includes the following steps.

[0044] Step 1: Obtain dual-phase CT pulmonary angiography images.

[0045] For the acquisition of dual-phase CT pulmonary angiography images, when performing angiography image scanning of the pulmonary artery phase on a patient, a pulmonary angiography image of the venous phase or the aortic phase can also be collected. That is, the dual-phase CT pulmonary angiography images in this embodiment include a three-dimensional pulmonary artery phase image of the pulmonary artery phase and a three-dimensional pulmonary vein phase image of the pulmonary vein phase.

[0046] Step 2: Construct an embolism area segmentation model based on a convolutional neural network, input the dual-phase CT pulmonary angiography images into the embolism area segmentation model for embolism detection and analysis to obtain dual-phase pulmonary embolism detection images.

[0047] Among them, the embolism area segmentation model is obtained by training based on a training data set with pulmonary embolism labeled images. Therefore, the embolism area segmentation model can be used for pulmonary embolism segmentation.

[0048] As Figure 4 shown, the embolism area segmentation model consists of n + 1 residual modules. Among them, n is a natural number greater than 1. The residual modules are divided into downsampling residual modules and upsampling residual modules. Specifically, among the n + 1 residual modules, the first to the n / 2th residual modules are downsampling residual modules; the (n / 2 + 1)th to the (n + 1)th residual modules are upsampling residual modules.

[0049] As Figure 5 shown, the downsampling residual module includes a first convolutional layer, a second convolutional layer, and a third convolutional layer. The first convolutional layer, the second convolutional layer, and the third convolutional layer are sequentially connected through the activation function Relu. That is, the third convolutional layer performs a downsampling convolutional operation twice. After the third convolutional layer is added to the input data of the downsampling residual module through the activation function Relu, downsampling binary segmentation data is obtained. Such a downsampling residual module makes the feature map size of the input image of the embolism area segmentation model become 1 / 2 of the original and the number of channels C become twice that of the previous layer every time it passes through a downsampling residual module.

[0050] As Figure 6As shown in the figure, the upsampling residual module includes a transposed convolution layer, a fourth convolution layer, and a fifth convolution layer. The transposed convolution layer, the fourth convolution layer, and the fifth convolution layer are sequentially connected through the activation function Relu. After the fifth convolution layer adds the input data of the upsampling residual module through the activation function Relu, upsampled binary segmentation data is obtained. Due to the setting of the transposed convolution layer, for the input image of the embolism area segmentation model, every time it passes through an upsampling residual module, the size of its feature map becomes twice the original, and the number of channels C becomes 1 / 2 of the previous layer.

[0051] Such an embolism area segmentation model makes it such that if the input is a CT pulmonary angiography image with a size of M*M, the output image size is also M*M, and the image is a binary segmentation image, which consists of binary segmentation data 1 and 0. Among them, the value of 1 represents the pulmonary embolism area, and the value of 0 represents the non-embolism area.

[0052] Step 3: Construct a registration training model for contrast images based on a convolutional neural network. Input the dual-phase pulmonary embolism detection images into the registration training model for contrast images for spatial transformation based on the displacement field to obtain the pulmonary artery embolism area detection image and the pulmonary vein embolism area detection image.

[0053] As Figure 7 shown, the registration training model for contrast images includes an encoding-decoding module and a spatial transformation module.

[0054] Among them, the encoding-decoding module consists of 9 convolutional modules and is used to input two standard three-dimensional pulse-phase images, such as the preprocessed three-dimensional pulmonary artery phase image and the three-dimensional pulmonary vein phase image. Among them, the preprocessing method is to truncate the CT value of the standard three-dimensional pulse-phase image and normalize it to the value between 0 and 1. The preprocessing method is to truncate the CT value of the standard three-dimensional pulse-phase image and normalize it to the value between 0 and 1. Perform downsampling encoding operations on these two standard three-dimensional pulse-phase images through the first 4 convolutional layer modules respectively, and then perform upsampling decoding through the last 4 convolutional layer modules to output the displacement field image.

[0055] The spatial transformation module is used to perform spatial transformation on all pixels in the three-dimensional pulmonary vein phase image among the two standard three-dimensional pulse-phase images according to the displacement field image to obtain the registered venous phase image.

[0056] In addition, the registration training model for contrast images also includes an iterative training module, which is used to continue the encoding-decoding and spatial transformation training on the registered venous phase image and the three-dimensional pulmonary artery phase image through iterative training, so as to minimize the difference between the finally registered pulmonary vein phase image and the three-dimensional pulmonary artery phase image, and complete the registration training.

[0057] As Figure 8As shown, after constructing the contrast image registration training model, the dual-phase pulmonary embolism detection image can be input into the contrast image registration training model for spatial transformation based on the displacement field. The specific operation is to use the dual-phase pulmonary embolism detection image as the image input of the spatial transformation module, and perform spatial transformation on the dual-phase pulmonary embolism detection image based on the displacement field image obtained from two standard three-dimensional pulse-phase images, so as to obtain the pulmonary artery embolism region detection image and the pulmonary vein embolism region detection image.

[0058] Step 4: According to the overlap ratio relationship between the isolated pulmonary embolism regions included in the pulmonary artery embolism region detection image and the pulmonary vein embolism region detection image, determine whether the isolated pulmonary embolism region is a positive embolism region.

[0059] Specifically, assume that the pulmonary artery embolism region detection image contains i isolated pulmonary artery embolism regions, numbered P 1 , P 2 ,......, P i-1 , P i ; the pulmonary vein embolism region detection image contains j isolated pulmonary vein embolism regions, numbered Q 1 , Q 2 ,......, Q i-1 , Q i . Then calculate the overlap ratio between the isolated pulmonary artery embolism region and the isolated pulmonary vein embolism region. If the overlap ratio is greater than or equal to a standard overlap ratio, such as 50% of the total area of the isolated pulmonary artery embolism region, then the embolism region is determined to be a positive embolism region. Otherwise, it is determined to be a false positive pulmonary embolism region.

[0060] Such as Figure 9 shown, the present invention also discloses an automatic pulmonary embolism detection system implemented by using the above-mentioned automatic pulmonary embolism detection method based on CT pulmonary angiography images, which includes an image acquisition module, an embolism detection module, an embolism region registration module, and an embolism confirmation module.

[0061] Among them, the image acquisition module is used to obtain dual-phase CT pulmonary angiography images; the embolism detection module is used to identify suspected embolism regions in the dual-phase pulmonary angiography images; the embolism region registration module is used for registering dual-phase pulmonary embolism detection images; the embolism confirmation module is used for comparing and confirming the embolism regions in the registered dual-phase pulmonary angiography images.

[0062] The above are only the preferred embodiments of the present invention. It should be noted that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which will not affect the implementation effect of the present invention and the practicality of the patent.

Claims

1. An automatic pulmonary embolism detection method based on CT pulmonary angiography images, characterized in that, it includes the following steps, Step 1: Obtain dual-phase CT pulmonary angiography images; Step 2: Construct an embolism area segmentation model based on a convolutional neural network, input the dual-phase CT pulmonary angiography images into the embolism area segmentation model for embolism detection and analysis to obtain dual-phase pulmonary embolism detection images; Step 3: Construct a contrast image registration training model based on a convolutional neural network; input the dual-phase pulmonary embolism detection images into the contrast image registration training model for spatial transformation based on the displacement field to obtain a pulmonary artery embolism area detection image and a pulmonary vein embolism area detection image; The contrast image registration training model includes, an encoding and decoding module, which consists of m convolutional modules, is used to input two standard three-dimensional pulse-phase images, perform downsampling encoding operations on the two standard three-dimensional pulse-phase images through the 1st to the m / 2th convolutional layer modules, and then perform upsampling decoding through the (m / 2 + 1)th to the mth convolutional layer modules to output a displacement field image; a spatial transformation module, which is used to perform spatial transformation on all pixels in the three-dimensional pulmonary vein phase image of the two standard three-dimensional pulse-phase images according to the displacement field image to obtain a registered venous phase image; wherein, m is a natural number greater than or equal to 1; Take the dual-phase pulmonary embolism detection image as the image input of the spatial transformation module, and perform spatial transformation on the dual-phase pulmonary embolism detection image based on the displacement field image obtained from the two standard three-dimensional pulse-phase images, so as to obtain a pulmonary artery embolism area detection image and a pulmonary vein embolism area detection image; Step 4: According to the overlap ratio relationship between the isolated pulmonary embolism areas included in the pulmonary artery embolism area detection image and the pulmonary vein embolism area detection image, judge whether the isolated pulmonary embolism area is a positive embolism area.

2. The automatic pulmonary embolism detection method based on CT pulmonary angiography images according to claim 1, characterized in that, the embolism area segmentation model consists of n + 1 residual modules; divide the residual modules into downsampling residual modules and upsampling residual modules; the downsampling residual module includes a first convolutional layer, a second convolutional layer and a third convolutional layer; the first convolutional layer, the second convolutional layer and the third convolutional layer are sequentially connected through an activation function Relu, and after the third convolutional layer is added to the input data of the downsampling residual module through the activation function Relu, downsampling binary segmentation data is obtained; the upsampling residual module includes a deconvolutional layer, a fourth convolutional layer and a fifth convolutional layer; the deconvolutional layer, the fourth convolutional layer and the fifth convolutional layer are sequentially connected through an activation function Relu, and after the fifth convolutional layer is added to the input data of the upsampling residual module through the activation function Relu, upsampling binary segmentation data is obtained; wherein, n is a natural number greater than or equal to 1.

3. The automatic pulmonary embolism detection method based on CT pulmonary angiography images according to claim 2, characterized in that, Among the n + 1 residual modules, the first to the n / 2-th residual modules are downsampling residual modules; the (n / 2 + 1)-th to the (n + 1)-th residual modules are upsampling residual modules.

4. An automatic pulmonary embolism detection method based on CT pulmonary angiography images according to claim 1, characterized in that, the registration training model of the angiography image further includes an iterative training module for performing encoding, decoding, and spatial transformation training on the registered venous phase image and the three-dimensional pulmonary artery phase image in the two standard three-dimensional pulse phase images through iterative training, so as to minimize the difference between the registered pulmonary venous phase image and the three-dimensional pulmonary artery phase image, and complete the registration training.

5. An automatic pulmonary embolism detection method based on CT pulmonary angiography images according to claim 4, characterized in that, the two standard three-dimensional pulse phase images are preprocessed three-dimensional pulmonary artery phase images and three-dimensional pulmonary vein phase images.

6. An automatic pulmonary embolism detection method based on CT pulmonary angiography images according to claim 5, characterized in that, the preprocessing method is to truncate the CT value of the standard three-dimensional pulse phase image and normalize it to a value between 0 and 1.

7. An automatic pulmonary embolism detection method based on CT pulmonary angiography images according to claim 1, characterized in that, mark the isolated pulmonary artery embolism region in the pulmonary artery embolism region detection image, and mark the isolated pulmonary vein embolism region in the pulmonary vein embolism region detection image; preset a standard overlap ratio; calculate the overlap ratio between the isolated pulmonary artery embolism region and the isolated pulmonary vein embolism region; if the overlap ratio is greater than or equal to the standard overlap ratio, it is determined that the embolism region is a positive embolism region; otherwise, it is determined as a false positive pulmonary embolism region.

Images