CT Image Processing Method, Processing Device and Electronic Device Based on Deep Convolutional Neural Network

Abstract

The present invention provides a CT image processing method, apparatus and electronic device based on a deep convolutional neural network. The processing method includes: acquiring an original thick-layer CT image including a patient's target part; performing bed removal and cropping processing on the original thick-layer CT image to obtain a cropped thick-layer CT image; performing normalization processing on the cropped thick-layer CT image; performing size adjustment and slicing processing on the thick-layer CT image according to the type of the patient's target part to obtain a thick-layer CT image with a predetermined size; inputting the thick-layer CT image into a trained deep convolutional neural network model to obtain a thin-layer CT prediction image; performing post-processing on the thin-layer CT prediction image to obtain a final thin-layer CT image. By means of a pre-trained deep convolutional neural network model, the present invention converts a thick-layer CT image into a high-quality thin-layer CT image, thereby improving the quality of the reconstructed DRR image and ultimately improving the accuracy and success rate of registration with an X-ray image.

Description

Technical Field

[0001] The present invention relates to the medical field, and in particular, to a CT image processing method, a processing device, and an electronic device based on a deep convolutional neural network. Background Art

[0002] The core of the IGPS image-guided positioning system is the two-dimensional to three-dimensional image registration of X-ray images and treatment plan CT images. Generally, a digitally reconstructed radiograph (DRR) is obtained from the CT image first, and then the two-dimensional to two-dimensional image registration of the X-ray image and the DRR image is implemented. The quality of the CT image directly determines the quality of the DRR image, and thus ultimately determines the registration effect and the registration success rate.

[0003] At present, most hospitals use thick-layer CT images with a thickness of 5 mm, 3 mm, etc. Such thick-layer CT images have a large interlayer distance and a low interlayer resolution. Therefore, the quality of the generated DRR images is poor, directly affecting the registration effect and the registration success rate with the X-ray images.

[0004] For the head and neck and pelvic regions, due to the obvious bone structure, the registration effect of the DRR image generated from such thick-layer CT images and the X-ray image is still acceptable. However, for the chest and abdomen, due to the unclear bone structure and the interference of respiratory movement, at this time, using the DRR image generated from such thick-layer CT images to register with the X-ray image greatly reduces the registration success rate and prolongs the time required for setup verification.

[0005] In the traditional radiotherapy process, for thick-layer CT images, methods based on gray scale or data interpolation between image tomograms based on shape and gray scale are mostly used. The CT images obtained by this method do not have a large improvement. Sometimes, distortion occurs, and more seriously, a sawtooth shape result appears. Moreover, the reconstructed DRR images do not have a large improvement in bone structure and texture, and sometimes a lot of artifacts are added. The accuracy and success rate of registration with X-ray images still need to be further improved. Summary of the Invention

[0006] In order to solve the problems existing in the traditional thick-layer CT image processing method, the present invention provides a CT image processing method, a processing device, and an electronic device based on a deep convolutional neural network.

[0007] The CT image processing method based on a deep convolutional neural network proposed by the present invention includes the following steps:

[0008] Obtain an original thick-layer CT image including the target part of the patient;

[0009] Perform decubitus removal and cropping on the original thick-layer CT image to obtain the cropped thick-layer CT image;

[0010] Perform normalization on the cropped thick-layer CT image;

[0011] Perform size adjustment and slicing on the thick-layer CT image according to the type of the patient's target part to obtain a thick-layer CT image with a predetermined size;

[0012] Input the thick-layer CT image into the trained deep convolutional neural network model corresponding to the predetermined size to obtain a thin-layer CT prediction image;

[0013] Perform post-processing on the thin-layer CT prediction image to obtain the final thin-layer CT image.

[0014] In some embodiments, performing decubitus removal and cropping on the original thick-layer CT image to obtain the cropped thick-layer CT image includes:

[0015] Calculate the optimal elliptical region of the original thick-layer CT image;

[0016] Perform image processing on the image of the optimal elliptical region to obtain multiple connected regions;

[0017] Integrate the multiple connected regions according to the preset threshold requirement to obtain a region, and obtain the decubitus-removed thick-layer CT image according to the integrated region;

[0018] Calculate the maximum contour boundary of the decubitus-removed thick-layer CT image;

[0019] Crop the decubitus-removed thick-layer CT image according to the maximum contour boundary to obtain the cropped thick-layer CT image.

[0020] In some embodiments, calculating the optimal elliptical region of the original thick-layer CT image includes: performing binarization on the original thick-layer CT image to obtain a binary image; performing edge detection and morphological operations on the binary image to obtain the largest binary image in the image set; obtaining the major axis, minor axis and the centroid of the largest binary image to obtain the optimal elliptical region;

[0021] Performing image processing on the image of the optimal elliptical region to obtain multiple connected regions includes: performing binarization on the image of the optimal elliptical region to obtain a binary image; performing processing on the binary image to obtain multiple connected regions.

[0022] In some embodiments, the patient's target part includes the head and neck, pelvis, and chest and abdomen, where:

[0023] When the target part of the patient is the head and neck or the pelvic part, size adjustment and slicing processing of the thick-layer CT image are performed according to the type of the target part of the patient to obtain a thick-layer CT image with a predetermined size, including:

[0024] Adjust the size of the cropped thick-layer CT image to the first size;

[0025] Perform slicing processing on the thick-layer CT image after size adjustment to obtain a first set of thick-layer CT images;

[0026] When the target part of the patient is the chest and abdomen, size adjustment and slicing processing of the thick-layer CT image are performed according to the type of the target part of the patient to obtain a thick-layer CT image with a predetermined size, including:

[0027] Adjust the size of the cropped thick-layer CT image to the first size;

[0028] Obtain a local image of the spinal region with a second size from the cropped thick-layer CT image;

[0029] Perform slicing processing on the thick-layer CT image after size adjustment and the obtained local image respectively to obtain a second set of thick-layer CT images and a third set of thick-layer CT images.

[0030] In some embodiments, the deep convolutional neural network model includes a first deep convolutional neural network model corresponding to the first-size image and a second deep convolutional neural network model corresponding to the second-size image;

[0031] When the target part of the patient is the head and neck or the pelvic part, input the thick-layer CT image into the trained deep convolutional neural network model corresponding to the predetermined size to obtain a thin-layer CT prediction image, including:

[0032] Input the first set of thick-layer CT images into the first deep convolutional neural network model to obtain a first thin-layer CT prediction image;

[0033] When the target part of the patient is the chest and abdomen, input the thick-layer CT image into the trained deep convolutional neural network model corresponding to the predetermined size to obtain a thin-layer CT prediction image, including:

[0034] Input the second set of thick-layer CT images into the first deep convolutional neural network model to obtain a second thin-layer CT prediction image;

[0035] Input the third set of thick-layer CT images into the second deep convolutional neural network model to obtain a third thin-layer CT prediction image.

[0036] In some embodiments, when the patient's target site is the head and neck or the pelvic region, post-processing is performed on the thin-slice CT prediction image to obtain the final thin-slice CT image, including:

[0037] Restore the size of the first thin-slice CT prediction image to be the same as the size of the cropped thick-slice CT image;

[0038] Perform denormalization processing on the first thin-slice CT prediction image after the size adjustment is completed;

[0039] Adjust the size of the first thin-slice CT prediction image after the denormalization processing is completed to be the same as the original thick-slice CT image, that is, obtain the final thin-slice CT image;

[0040] When the patient's target site is the chest and abdomen, post-processing is performed on the thin-slice CT prediction image to obtain the final thin-slice CT image, including:

[0041] Restore the size of the second thin-slice CT prediction image to be the same as the size of the cropped thick-slice CT image;

[0042] Use the third thin-slice CT prediction image to replace the image in the corresponding region of the second thin-slice CT prediction image to obtain a fused thin-slice CT prediction image;

[0043] Perform denormalization processing on the fused thin-slice CT prediction image;

[0044] Adjust the size of the fused thin-slice CT prediction image after the denormalization processing is completed to be the same as the original thick-slice CT image, that is, obtain the final thin-slice CT image.

[0045] In some embodiments, the first size is 256×256 and the second size is 128×128.

[0046] The CT image processing device based on a deep convolutional neural network proposed by the present invention includes:

[0047] An image acquisition module, configured to acquire the original thick-slice CT image including the patient's target site;

[0048] A bed removal and cropping processing module, configured to perform bed removal and cropping processing on the original thick-slice CT image to obtain a cropped thick-slice CT image;

[0049] A normalization processing module, configured to perform normalization processing on the cropped thick-slice CT image;

[0050] A size adjustment and slicing processing module, configured to perform size adjustment and slicing processing on the thick-slice CT image according to the type of the patient's target site to obtain a thick-slice CT image with a predetermined size;

[0051] A prediction module for inputting thick-layer CT images into a pre-trained deep convolutional neural network model corresponding to a predetermined size to obtain thin-layer CT prediction images;

[0052] A post-processing module for post-processing the thin-layer CT prediction images to obtain the final thin-layer CT images.

[0053] The electronic device proposed by the present invention includes a memory, a processor, and a computer program stored in the memory and operable on the processor. Among them, when the processor executes the program, it implements the CT image processing method based on a deep convolutional neural network described in any one of the above.

[0054] The above CT image processing method, device, and electronic device based on a deep convolutional neural network convert thick-layer CT images into high-quality thin-layer CT images through a pre-trained deep convolutional neural network model, thereby improving the quality of the reconstructed DRR images, and ultimately improving the accuracy and success rate of registration with X-ray images.

[0055] In particular, for different patient target parts (head and neck, pelvis, and chest and abdomen), the present invention adopts different processing procedures to obtain thin-layer CT images, thereby ensuring that: for different patient target parts, the thin-layer CT images formed by the present invention can all form high-quality DDR images, and can effectively improve the image processing efficiency and save system resources. Description of the Drawings

[0056] Figure 1 It is the execution flowchart of the CT image processing method based on a deep convolutional neural network in the embodiment of the present invention;

[0057] Figure 2 It is the execution flowchart of performing bed removal and cropping processing on the original thick-layer CT image in the embodiment of the present invention;

[0058] Figure 3 It is a schematic diagram of the specific implementation process of converting thick-layer CT images into thin-layer CT images in the embodiment of the present invention;

[0059] Figure 4 It is a schematic diagram of the specific implementation process of post-processing the thin-layer CT images in the embodiment of the present invention;

[0060] Figure 5 It is the architecture diagram of the pre-trained deep convolutional neural network model in the embodiment of the present invention;

[0061] Figure 6 It is a schematic diagram of the training process of the deep convolutional neural network model in the embodiment of the present invention;

[0062] Figure 7Schematic diagram of the specific training process of the deep convolutional neural network model in the embodiments of the present invention;

[0063] Figure 8 Comparison diagram of the head and neck DDR reconstructed from the original thick-layer CT image and the head and neck DDR reconstructed from the thin-layer CT image in the embodiments of the present invention;

[0064] Figure 9 Comparison diagram of the chest and abdomen DDR in the supine position reconstructed from the original thick-layer CT image and the chest and abdomen DDR in the supine position reconstructed from the thin-layer CT image in the embodiments of the present invention;

[0065] Figure 10 Comparison diagram of the lower abdomen DDR in the supine position reconstructed from the original thick-layer CT image and the lower abdomen DDR in the supine position reconstructed from the thin-layer CT image in the embodiments of the present invention;

[0066] Figure 11 Comparison diagram of the pelvic DDR reconstructed from the original thick-layer CT image and the pelvic DDR reconstructed from the thin-layer CT image in the embodiments of the present invention;

[0067] Figure 12 Structural block diagram of the CT image processing device in the embodiments of the present invention;

[0068] Figure 13 Structural block diagram of the electronic device in the embodiments of the present invention. Detailed implementation manners

[0069] It should be pointed out that the content described in the following detailed description is all exemplary, and the purpose is to give an indicative description of the content of the present invention. It should be noted that all technical and scientific terms used in the present invention have the same meaning as commonly understood by those of ordinary skill in the technical field to which the invention belongs.

[0070] Next, in combination with the accompanying drawings in the embodiments of the present invention, the system architecture in the embodiments of the present invention and the solutions in the prior art will be clearly and completely described. It should be noted that the described embodiments are only for explaining and illustrating the present invention, rather than all the content. Based on the embodiments provided by the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention application.

[0071] Method embodiments

[0072] As Figure 1 shown, the embodiments of the present invention propose a CT image processing method based on a deep convolutional neural network, which includes the following steps:

[0073] S100: Obtain the original thick-layer CT image including the target part of the patient.

[0074] S200: Perform decubitus removal and cropping on the original thick-slice CT image to obtain the cropped thick-slice CT image.

[0075] S300: Perform normalization on the cropped thick-slice CT image.

[0076] S400: Perform size adjustment and slicing on the thick-slice CT image according to the type of the patient's part to obtain a thick-slice CT image with a predetermined size.

[0077] S500: Input the thick-slice CT image into the trained deep convolutional neural network model corresponding to the predetermined size to obtain a thin-slice CT prediction image.

[0078] S600: Perform post-processing on the thin-slice CT prediction image to obtain the final thin-slice CT image.

[0079] The above CT image processing method based on a deep convolutional neural network converts a thick-slice CT image into a high-quality thin-slice CT image through a pre-trained deep convolutional neural network model, thereby improving the quality of the reconstructed DRR image and ultimately increasing the accuracy and success rate of registration with the X-ray image.

[0080] In particular, for different patient target parts (head and neck, pelvis, and chest and abdomen), the present invention uses different processing procedures to obtain thin-slice CT images, thereby ensuring that: for different patient target parts, the thin-slice CT images formed by the present invention can all form high-quality DDR images, and can effectively improve the image processing efficiency and save system resources.

[0081] Reference Figure 2 and Figure 3 As shown, optionally, the performing decubitus removal and cropping on the original thick-slice CT image in step S200 to obtain the cropped thick-slice CT image may specifically include the following steps:

[0082] S210: Calculate the best elliptical region of the original thick-slice CT image.

[0083] Specifically, it may include:

[0084] S211: Perform binarization on the original thick-slice CT image to obtain a binary image. Optionally, the original thick-slice CT image may be binarized using a predetermined first threshold to obtain a binary image. For example, the first threshold may be 800.

[0085] S212: Perform opening operation on the binary image.

[0086] S213. Perform Sobel edge detection and closing operation on the binary image after opening operation to obtain the closed contour of the region in the binary image.

[0087] S214. Perform hole filling on the binary image after obtaining the closed contour to obtain the largest binary image in the image set.

[0088] It should be understood that opening operation is performed on the obtained binary image to eliminate the noise points in the binary image. Then, edge detection processing is performed on the above obtained result to obtain the closed contour of the region in the binary image. Since the contour obtained by edge detection is not necessarily closed, closing operation is performed later to further ensure that the obtained contour is closed. Hole filling is performed on the closed contour to obtain the largest binary image.

[0089] S215. Calculate the major axis, minor axis and centroid of the largest binary image to obtain the best elliptical region.

[0090] Optionally, according to the largest binary image obtained by the above processing, calculate its regional centroid. Subsequently, obtain the minimum circumscribed rectangle of the above largest binary image, determine the major and minor axes a and b, and compare them with the major and minor axis thresholds A (such as 245) and B (such as 200). If less than the threshold, a and b are retained; if greater than the threshold, a and b are respectively assigned A and B to remove the redundant bed board and reduce the post-processing calculation amount.

[0091] More specifically, it includes:

[0092] Calculate the centroid of the largest binary image;

[0093] Obtain the minimum circumscribed rectangle of the largest binary image;

[0094] Compare according to the minimum circumscribed rectangle and the thresholds of the major and minor axes to determine the major axis and minor axis of the best elliptical region;

[0095] Obtain the best elliptical region according to the major axis, minor axis and the centroid of the largest binary image.

[0096] Perform elliptical region processing on the image according to the obtained centroid and major and minor axes a, b, retain the image inside the ellipse, and set the outside of the ellipse to 0, that is, most of the bed board is removed.

[0097] S220. Perform image processing on the image of the best elliptical region to obtain multiple connected regions.

[0098] Specifically, it may include:

[0099] S221. Perform binarization on the image of the best elliptical region to obtain a binary image.

[0100] In an embodiment of the present invention, the best elliptical region is binarized using a predetermined second threshold to obtain a binary image. Of course, the value of the first threshold can also be used as the second threshold. For example, the second threshold can also be set to 800. S222. Process the binary image to obtain a plurality of connected regions.

[0101] Optionally, a plurality of connected regions are obtained by performing edge detection and "cross" processing on the binary image, specifically including:

[0102] Perform Canny edge detection on the binary image to obtain an edge detection result.

[0103] Perform "cross" processing on the edge detection result to connect the discontinuous points to obtain a plurality of processed regions.

[0104] Judge whether the edges of each processed region are connected.

[0105] If so, perform closing operation and hole filling processing on the processed region to obtain a plurality of connected regions.

[0106] If not, perform closing operation and hole filling processing after closing the unclosed places on the edge of the processed region.

[0107] It should be understood that the obtained image is binarized according to the second threshold to obtain a binary image, and connected regions are obtained through edge detection and "cross" processing. To prevent the situation that individual images exceed the image size and cause no connected regions, it is necessary to judge the edge detection image, close the continuously unclosed places, and perform closing operation and hole filling on it.

[0108] S230. Integrate a plurality of connected regions into one region according to a preset threshold requirement, and obtain a thick-layer CT image after removing the bed according to the integrated region.

[0109] Specifically including:

[0110] Compare each connected region with an area threshold;

[0111] Integrate all connected regions that satisfy the area being greater than the area threshold into one region.

[0112] Obtain a thick-layer CT image after removing the bed according to the integrated region.

[0113] It should be understood that the connected regions are screened by the aspect ratio (L1, L2) (for example, L1 is taken as 0.3 and L2 is taken as 3) and the area threshold D (for example, taken as 1200), the connected regions with an area greater than D are integrated into one region, and after performing an AND operation on the integrated region and the image after obtaining the best elliptical region, a thick-layer CT image after removing the bed is obtained.

[0114] S240. Calculate the maximum contour boundary of the thick-layer CT image set after removing the bed.

[0115] S250. Crop the thick-layer CT image after removing the bed according to the maximum contour boundary to obtain the cropped thick-layer CT image.

[0116] After the bed removal and cropping processing in step S200 and the normalization processing in step S300, the format of the original thick-layer CT image basically meets the input requirements of the deep convolutional neural network model.

[0117] As described in the background art of the present invention, among the three main patient target parts, the image registration of the chest and abdomen has much higher requirements for the quality of CT images than the head and neck and the pelvis.

[0118] Therefore, in the implementation of the present invention, there are significant differences between the CT image processing processes for the chest and abdomen and those for the head and neck and the pelvis in steps S400 to S500. Specifically as follows:

[0119] According to different situations, the execution process of the size adjustment and slicing processing of the thick-layer CT image in step S400 is as follows:

[0120] Situation 1. For the patient target part being the head and neck or the pelvis, step S400 specifically includes:

[0121] Adjust the size of the cropped thick-layer CT image to a larger first size, such as 256×256.

[0122] Perform slicing processing on the thick-layer CT image after the size adjustment to obtain the first thick-layer CT image set. Among them, the slicing processing is to insert blank black images between the images of the thick-layer CT image.

[0123] That is to say, the thick-layer CT images of the head and neck or the pelvis are only adjusted to a unified image set with a larger first size.

[0124] Situation 2. For the patient target part being the chest and abdomen, step S400 specifically includes:

[0125] Adjust the size of the cropped thick-layer CT image to the first size, such as 256×256.

[0126] Obtain a local image of the spinal region with a second size from the cropped thick-layer CT image, where the second size is smaller than the first size, such as 128×128.

[0127] Next, perform slicing processing on the thick-layer CT images with adjusted dimensions and the obtained local images to obtain a second set of thick-layer CT images and a third set of thick-layer CT images. Among them, slicing processing means inserting blank black images between the images of the thick-layer CT images.

[0128] That is to say, the thick-layer CT images of the chest and abdomen are adjusted to have two image sets with different dimensions. Among them, the second set of thick-layer CT images shows the overall view of the chest and abdomen, while the third set of thick-layer CT images shows the local content of the spinal region. That is: the images of the spinal region are also extracted separately.

[0129] Continue to refer to Figure 3 As shown, the optional implementation process of obtaining the local image of the spinal region with the second dimension from the cropped thick-layer CT images is as follows:

[0130] First, judge the patient's body position according to the gray value threshold D (such as taking 1000), so as to know whether the spine is in the upper half or the lower half of the image. If the spine is in the upper half of the image, starting from the starting position of the image, search for the 128×128 area containing the most spine parts with step = 10 for the gray value threshold E (such as taking 1200). Otherwise, starting from the middle row position of the image, search for the 128×128 area containing the most spine parts with step = 10 for the gray value threshold E (such as 1200).

[0131] In particular, in order to be able to process the CT images of the first dimension and the CT images of the second dimension, the embodiments of the present invention need to pre-train two different deep convolutional neural network models, namely the first deep convolutional neural network model corresponding to the first-dimension images and the second deep convolutional neural network model corresponding to the second-dimension images.

[0132] According to different situations, the execution process of predicting the thin-layer CT prediction image in step S500 is as follows:

[0133] Situation 1: For the patient's target part being the head and neck or the pelvic part, step 500 specifically includes:

[0134] Input the first set of thick-layer CT images into the first deep convolutional neural network model to obtain the first thin-layer CT prediction image.

[0135] For example, if the image size of the first set of thick-layer CT images is 256×256, then the image size of the first thin-layer CT prediction image for the head and neck or the pelvic part output is 256×256.

[0136] Situation 2: For the patient's target part being the chest and abdomen, step 500 specifically includes:

[0137] Input the second thick-layer CT image set into the first deep convolutional neural network model to obtain the second thin-layer CT prediction image.

[0138] Input the third thick-layer CT image set into the second deep convolutional neural network model to obtain the third thin-layer CT prediction image.

[0139] For example, if the image size of the second thick-layer CT image set is 256×256 and the image size of the third thick-layer CT image set is 128×128, then two sets of thin-layer CT prediction images for the chest and abdomen are output, where

[0140] The second thin-layer CT prediction image with a size of 256×256 is a panoramic image of the chest and abdomen.

[0141] The third thin-layer CT prediction image with a size of 128×128 is a local image of the spinal region.

[0142] According to different situations, the post-processing of the thin-layer CT prediction image in step S600 is performed as follows:

[0143] Situation 1: For the patient's target area being the head and neck or pelvis, step S600 specifically includes:

[0144] Restore the size of the first thin-layer CT prediction image to be the same as the size of the cropped thick-layer CT image.

[0145] Perform denormalization processing on the first thin-layer CT prediction image after the size adjustment is completed.

[0146] Adjust the size of the first thin-layer CT prediction image after the denormalization processing is completed to be the same as the original thick-layer CT image to obtain the final thin-layer CT image.

[0147] Situation 2: For the patient's target area being the chest and abdomen, step S600 specifically includes:

[0148] Restore the size of the second thin-layer CT prediction image to be the same as the size of the cropped thick-layer CT image.

[0149] Use the third thin-layer CT prediction image to replace the image of the corresponding spinal region in the second thin-layer CT prediction image to obtain the fused thin-layer CT prediction image. That is, integrate the thin-layer CT prediction image of the spinal region into the corresponding area of the panoramic image of the chest and abdomen, so that the image quality of the spinal region is significantly improved, and perfectly solves the technical problem of "in the chest and abdomen, due to the unclear bone structure and the interference of respiratory movement, the image quality is not high" mentioned in the background technology.

[0150] Perform denormalization processing on the fused thin-layer CT prediction image.

[0151] Adjust the size of the fused thin-slice CT prediction image after inverse normalization to be the same as that of the original thick-slice CT image, that is, obtain the final thin-slice CT image.

[0152] Figures 8 to 11 Respectively shown are the comparison diagrams of the head and neck DDR reconstructed from the original thick-slice CT image and the head and neck DDR reconstructed from the thin-slice CT image of the embodiment of the present invention, the comparison diagrams of the chest and abdomen DDR in the supine position reconstructed from the original thick-slice CT image and the chest and abdomen DDR in the supine position reconstructed from the thin-slice CT image of the embodiment of the present invention, the comparison diagrams of the lower abdomen DDR in the supine position reconstructed from the original thick-slice CT image and the lower abdomen DDR in the supine position reconstructed from the thin-slice CT image of the embodiment of the present invention, and the comparison diagrams of the pelvic DDR reconstructed from the original thick-slice CT image and the pelvic DDR reconstructed from the thin-slice CT image of the embodiment of the present invention.

[0153] From Figures 8 to 11 It can be clearly seen that the CT image processing method based on a deep convolutional neural network provided by the embodiment of the present invention significantly improves the quality of the reconstructed DRR image.

[0154] For convenience of use, the first deep convolutional neural network model and the second deep convolutional neural network model used in the embodiment of the present invention adopt deep convolutional neural networks with the same structure. Of course, the specific model parameters of the two are different. It should be noted that in the implementation of the present invention, various known and suitable deep convolutional neural networks can be selected.

[0155] For example, optionally, the deep convolutional neural network adopted in the embodiment of the present invention includes a first cascaded convolution operation, a pooling operation, an upsampling operation, a second cascaded convolution operation, and a third convolution operation, where:

[0156] The first cascaded convolution operation includes a first convolution operation and a first dilated convolution operation; the first convolution operation extracts features from the thick-slice CT image set through a plurality of convolution kernels to obtain a first convolution operation feature map; the first dilated convolution operation extracts features from the thick-slice CT image set through a plurality of dilated convolution kernels to obtain a first dilated convolution operation feature map; the first cascaded convolution operation is further used to perform feature combination on the first convolution operation feature map and the first dilated convolution operation feature map to obtain a first cascaded convolution operation feature map.

[0157] The pooling operation extracts features from the first cascaded convolution operation feature map to obtain a pooling operation feature map.

[0158] The upsampling operation uses an interpolation method to perform restoration processing on the pooling operation feature map to obtain an upsampling operation restoration map.

[0159] The second cascaded convolution operation includes a second convolution operation and a second dilated convolution operation; the second convolution operation extracts features from the upsampled and restored image through a number of convolutional kernels to obtain a second convolution operation feature map; the second dilated convolution operation extracts features from the upsampled and restored image through a number of dilated convolutional kernels to obtain a second dilated convolution operation feature map; the second cascaded convolution operation is also used to extract features from the second convolution operation feature map and the second dilated convolution operation feature map to obtain a second cascaded convolution operation feature map;

[0160] The third convolution operation is used to extract features from the second cascaded convolution operation feature map to obtain a thin-slice CT prediction image.

[0161] For example, in the first convolution operation, each convolution feature map extracts features from the thick-slice CT image set through a 3×3×3 convolutional kernel. As the network depth increases, the extracted features change from simple to complex. That is, the first convolution operation can adopt a 3D convolutional neural network. Using a 3D convolutional neural network can effectively utilize the three-dimensional information of the thick-slice CT image, thereby improving the accuracy of predicting the thin-slice CT prediction image.

[0162] For example, in the first dilated convolution operation, each dilated convolution feature map extracts features from the thick-slice CT image set through a 5×5×5 dilated convolutional kernel. Similarly, as the network depth increases, the extracted features also change from simple to complex. That is, the first dilated convolution operation can adopt a dilated convolutional neural network. When extracting features from the thick-slice CT image set, it can directly train the areas with images in the three-dimensional image, and the blank black images inserted between images do not participate in the training, thereby reducing the training parameters.

[0163] The first cascaded convolution operation performs feature concatenation on the first convolution operation feature map and the first dilated convolution operation feature map. That is, the first cascaded convolution operation adopts a cascaded neural network that cascades a convolutional neural network and a dilated convolutional neural network. It can effectively combine the feature maps obtained by both to increase image features. And compared with the copy function, the concatenation function has its own training parameters instead of sharing training parameters with the convolution, thereby increasing the image information and further improving the prediction accuracy.

[0164] Among them, since the pooling feature map uniquely corresponds to a first cascaded convolution operation feature map, the number of feature maps of the pooling operation will not change. The pooling operation has the function of secondary feature extraction and obtains features with spatial invariance by reducing the resolution of the feature map.

[0165] Among them, the upsampling operation uses an interpolation method to restore the feature map of the pooling operation, gradually restoring the details and size of the image to obtain the restored map of the upsampling operation. Further, convolution processing is performed through the second cascaded convolution operation and the third convolution operation, thereby obtaining the thin-slice CT prediction image. Compared with the unpooling operation, the cascaded neural network model using the upsampling operation can increase the image information, thereby improving the prediction accuracy.

[0166] As Figure 5 shown, optionally, the deep convolutional neural network of the embodiment of the present invention adopts an end-to-end neural network architecture in which the first cascaded convolution operation, the pooling operation, the upsampling operation, the second cascaded convolution operation, and the third convolution operation are interleaved. Among them, the black square represents the max pooling layer, and the juxtaposed white squares with black shadows respectively represent the convolution layer and the dilated convolution layer. The juxtaposed white squares respectively represent the cascaded convolution feature map and the upsampling layer. The number of feature maps is marked above the squares.

[0167] Among them, Figure 5 the left side is the encoder part, and the right side is the decoder part. The encoder part consists of a convolution layer, a dilated convolution layer, and a pooling layer. Among them, the convolution layer uses multiple 3×3×3 convolution kernels for feature extraction, and adopts local connection and weight sharing to deepen the network to effectively reduce the complexity of the network and reduce the number of training parameters; the dilated convolution layer also consists of multiple feature maps, and each feature map uses a 5×5×5 dilated convolution kernel for feature extraction. Subsequently, the feature maps obtained by the convolution operation and the dilated convolution operation are concatenated to obtain the feature map of the first cascaded convolution operation, and then the second feature extraction is performed on the feature map of the first cascaded operation through the max pooling layer, and the size of the feature map is reduced by half, which plays a role in reducing the calculation amount and memory consumption. The decoder part consists of a convolution layer, a dilated convolution layer, and an upsampling layer. Among them, the upsampling layer gradually restores the details and size of the image. Compared with unpooling, more image information can be obtained. In addition, feature combination is performed by concatenate. Compared with copy, the concatenate operation has its own training parameters, so that the network can more easily reconstruct the details of the image, while the former shares the training parameters with the convolution processing of the encoder part.

[0168] As Figures 6 to 7 shown, optionally, in the embodiment of the present invention, the specific training method of the deep convolutional neural network model includes:

[0169] S1. Obtain thick-layer CT images and thin-layer CT images of corresponding sizes. For example, when training the first deep convolutional neural network model, the sizes of the thick-layer CT images and thin-layer CT images obtained are 256×256, while when training the second deep convolutional neural network model, the sizes of the thick-layer CT images and thin-layer CT images obtained are 128×128.

[0170] S2. Perform slicing processing on the thick-layer CT images to obtain a thick-layer CT image set.

[0171] S3. Use the deep convolutional neural network model to train the thick-layer CT image set and the thin-layer CT image set to obtain the optimal model parameters of the deep convolutional neural network model.

[0172] S4. Output the optimal model parameters.

[0173] Among them, the slice thickness of the thick-layer CT images can be 4 mm, 3 mm, etc., and the slice thickness of the thin-layer CT images can be 2.0 mm, 1.5 mm, etc. The slicing processing is to insert blank black images between the images of the thick-layer CT images to ensure that the formed thick-layer CT image set has the same slices as the thin-layer CT images.

[0174] Optionally, using the deep convolutional neural network model to train the thick-layer CT image set and the thin-layer CT image set specifically includes:

[0175] S31. Perform iterative feature extraction and prediction on the thick-layer CT image set through the deep convolutional neural network model to obtain thin-layer CT prediction images.

[0176] S32. Perform error processing on the thin-layer CT prediction images and the thin-layer CT images through a loss function to obtain thin-layer CT prediction image error data.

[0177] S33. Perform backpropagation on the thin-layer CT prediction image error data through an optimization function.

[0178] Optionally, perform error processing on the thin-layer CT prediction images and the thin-layer CT images through a loss function to obtain thin-layer CT prediction image error data.

[0179] Optionally, use the MAE function (mean absolute error) as the loss function to obtain error data. The MAE function is:

[0180]

[0181] where Y i is the thin-layer CT image, and P i is the thin-layer CT prediction image.

[0182] Optionally, the error data of the thin-slice CT prediction image can be backpropagated through an optimization function. Optionally, the Adam stochastic optimization function is applied for backpropagation, which can be used to further optimize the parameters of the deep convolutional neural network model.

[0183] Device Embodiment

[0184] As Figure 12 shown, the CT image processing device based on a deep convolutional neural network provided by an embodiment of the present invention includes:

[0185] An image acquisition module 100, configured to acquire an original thick-slice CT image including a target part of a patient;

[0186] A bed removal and cropping processing module 200, configured to perform bed removal and cropping processing on the original thick-slice CT image to obtain a cropped thick-slice CT image;

[0187] A normalization processing module 300, configured to perform normalization processing on the cropped thick-slice CT image;

[0188] A size adjustment and slicing processing module 400, configured to perform size adjustment and slicing processing on the thick-slice CT image according to the type of the target part of the patient to obtain a thick-slice CT image with a predetermined size;

[0189] A prediction module 500, configured to input the thick-slice CT image into a trained deep convolutional neural network model corresponding to the predetermined size to obtain a thin-slice CT prediction image;

[0190] A post-processing module 600, configured to perform post-processing on the thin-slice CT prediction image to obtain a final thin-slice CT image.

[0191] Since the processing procedures of the functional modules of the CT image processing device based on a deep convolutional neural network provided by an embodiment of the present invention are consistent with the processing procedures of the CT image processing method based on a deep convolutional neural network provided by the foregoing method embodiment, the detailed processing procedures of the functional modules of the CT image processing device based on a deep convolutional neural network will not be repeatedly described in this embodiment, and the relevant content in the foregoing method embodiment can be directly referred to.

[0192] Device Embodiment

[0193] As Figure 13 shown, the electronic device provided by an embodiment of the present invention includes a processor 21 and a memory 23, and the processor 21 and the memory 23 are connected, such as connected through a bus 22.

[0194] The processor 21 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable devices, transistor logic devices, hardware components, or any other combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in connection with the disclosure of the present application. The processor 21 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0195] The bus 22 may include a path for transmitting information between the above components. The bus 22 may be a PCI bus, an EISA bus, or the like. The bus 22 may be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0196] The memory 23 may be a ROM or other type of static storage device that can store static information and instructions, a RAM, or other type of dynamic storage device that can store information and instructions. It may also be an EEPROM, a CD-ROM, or other optical disc storage, optical disc storage, magnetic disk storage medium, or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0197] The memory 23 is used to store the application program code of the solution of the present application and is controlled by the processor 21 for execution. The processor 21 is used to execute the application program code stored in the memory 23 to implement the CT image processing method based on a deep convolutional neural network provided in the foregoing method embodiments.

[0198] The embodiments of the present application finally also provide a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the CT image processing method based on a deep convolutional neural network provided in the foregoing method embodiments.

[0199] The above has described the present invention in sufficient detail with a certain degree of particularity. Those of ordinary skill in the art should understand that the descriptions in the embodiments are merely exemplary, and all changes made without departing from the true spirit and scope of the present invention should fall within the protection scope of the present invention. The scope of protection required by the present invention is defined by the claims described above, rather than by the above descriptions in the embodiments.

Claims

1. A CT image processing method based on a deep convolutional neural network, characterized in that, It includes: Obtain the original thick-layer CT image including the patient's target part; Perform bed removal and cropping processing on the original thick-layer CT image to obtain the cropped thick-layer CT image; Perform normalization processing on the cropped thick-layer CT image; According to the type of the patient's target part, perform size adjustment and slicing processing on the thick-layer CT image to obtain a thick-layer CT image with a predetermined size; Input the thick-layer CT image into the trained deep convolutional neural network model corresponding to the predetermined size to obtain a thin-layer CT prediction image; Perform post-processing on the thin-layer CT prediction image to obtain the final thin-layer CT image; The patient's target part includes the head and neck, pelvis, and chest and abdomen, where: When the patient's target part is the head and neck or pelvis, the step of performing size adjustment and slicing processing on the thick-layer CT image according to the type of the patient's target part to obtain a thick-layer CT image with a predetermined size includes: Adjust the size of the cropped thick-layer CT image to a first size; Perform slicing processing on the thick-layer CT image after size adjustment to obtain a first set of thick-layer CT images; When the patient's target part is the chest and abdomen, the step of performing size adjustment and slicing processing on the thick-layer CT image according to the type of the patient's target part to obtain a thick-layer CT image with a predetermined size includes: Adjust the size of the cropped thick-layer CT image to a first size; Obtain a local image of the spinal region with a second size from the cropped thick-layer CT image; Perform slicing processing on the thick-layer CT image after size adjustment and the obtained local image respectively to obtain a second set of thick-layer CT images and a third set of thick-layer CT images.

2. The CT image processing method based on a deep convolutional neural network according to claim 1, characterized in that, The step of performing bed removal and cropping processing on the original thick-layer CT image to obtain the cropped thick-layer CT image includes: Calculate the best elliptical region of the original thick-layer CT image; Perform image processing on the image of the best elliptical region to obtain multiple connected regions; Integrate multiple connected regions according to the preset threshold requirement to obtain one region, and obtain the thick-layer CT image after bed removal according to the integrated region; Calculate the maximum contour boundary of the thick-layer CT image after bed removal; Crop the thick-layer CT image after bed removal according to the maximum contour boundary to obtain the cropped thick-layer CT image.

3. The CT image processing method based on a deep convolutional neural network according to claim 2, wherein The step of calculating the best elliptical region of the original thick-layer CT image includes: Perform binarization processing on the original thick-layer CT image to obtain a binary image; Perform edge detection and morphological operations on the binary image to obtain the largest binary image in the image set; Obtain the major axis, minor axis, and centroid of the largest binary image to obtain the best elliptical region; The step of performing image processing on the image of the best elliptical region to obtain multiple connected regions includes: Perform binarization processing on the image of the best elliptical region to obtain a binary image; Perform processing on the binary image to obtain multiple connected regions.

4. The CT image processing method based on a deep convolutional neural network according to claim 1, wherein: The deep convolutional neural network model includes a first deep convolutional neural network model corresponding to an image of a first size and a second deep convolutional neural network model corresponding to an image of a second size; When the target part of the patient is the head and neck or the pelvic part, the step of inputting the thick-layer CT image into the trained deep convolutional neural network model corresponding to the predetermined size to obtain a thin-layer CT prediction image includes: Inputting the first set of thick-layer CT images into the first deep convolutional neural network model to obtain a first thin-layer CT prediction image; When the target part of the patient is the chest and abdomen, the step of inputting the thick-layer CT image into the trained deep convolutional neural network model corresponding to the predetermined size to obtain a thin-layer CT prediction image includes: Inputting the second set of thick-layer CT images into the first deep convolutional neural network model to obtain a second thin-layer CT prediction image; Inputting the third set of thick-layer CT images into the second deep convolutional neural network model to obtain a third thin-layer CT prediction image.

5. The CT image processing method based on a deep convolutional neural network according to claim 4, wherein: When the target part of the patient is the head and neck or the pelvic part, the step of post-processing the thin-layer CT prediction image to obtain a final thin-layer CT image includes: Restoring the size of the first thin-layer CT prediction image to be the same as the size of the cropped thick-layer CT image; Performing denormalization processing on the first thin-layer CT prediction image after the size adjustment is completed; Adjusting the size of the first thin-layer CT prediction image after the denormalization processing is completed to be the same as the size of the original thick-layer CT image to obtain the final thin-layer CT image; When the target part of the patient is the chest and abdomen, the step of post-processing the thin-layer CT prediction image to obtain a final thin-layer CT image includes: Restoring the size of the second thin-layer CT prediction image to be the same as the size of the cropped thick-layer CT image; Replacing the image in the corresponding area of the second thin-layer CT prediction image with the third thin-layer CT prediction image to obtain a fused thin-layer CT prediction image; Performing denormalization processing on the fused thin-layer CT prediction image; Adjusting the size of the fused thin-layer CT prediction image after the denormalization processing is completed to be the same as the size of the original thick-layer CT image to obtain the final thin-layer CT image.

6. The CT image processing method based on a deep convolutional neural network according to claim 1, characterized in that: The first size is 256×256, and the second size is 128×128.

7. The CT image processing method based on a deep convolutional neural network according to claim 1, wherein the deep convolutional neural network model includes a first cascaded convolution operation, a pooling operation, an upsampling operation, a second cascaded convolution operation, and a third convolution operation, where: The first cascaded convolution operation includes a first convolution operation and a first dilated convolution operation; the first convolution operation extracts features from the thick-layer CT image through a plurality of convolutional kernels to obtain a first convolution operation feature map; the first dilated convolution operation extracts features from the thick-layer CT image through a plurality of dilated convolutional kernels to obtain a first dilated convolution operation feature map; the first cascaded convolution operation is further configured to jointly extract features from the first convolution operation feature map and the first dilated convolution operation feature map to obtain a first cascaded convolution operation feature map; The pooling operation extracts features from the first cascaded convolution operation feature map to obtain a pooling operation feature map; The upsampling operation performs a restoration process on the pooling operation feature map using an interpolation method to obtain an upsampling operation restored map; The second cascaded convolution operation includes a second convolution operation and a second dilated convolution operation; the second convolution operation extracts features from the upsampling restored map through a plurality of convolutional kernels to obtain a second convolution operation feature map; the second dilated convolution operation extracts features from the upsampling restored map through a plurality of dilated convolutional kernels to obtain a second dilated convolution operation feature map; the second cascaded convolution operation is further configured to extract features from the second convolution operation feature map and the second dilated convolution operation feature map to obtain a second cascaded convolution operation feature map; The third convolution operation is configured to extract features from the second cascaded convolution operation feature map to obtain the thin-layer CT prediction image.

8. A CT image processing device based on a deep convolutional neural network, characterized in that, It includes: An image acquisition module, configured to acquire an original thick-layer CT image including a patient's target part; A bed removal and cropping processing module, configured to perform bed removal and cropping processing on the original thick-layer CT image to obtain a cropped thick-layer CT image; A normalization processing module, configured to perform normalization processing on the cropped thick-layer CT image; A size adjustment and slicing processing module, configured to perform size adjustment and slicing processing on the thick-layer CT image according to the type of the patient's target part to obtain a thick-layer CT image with a predetermined size; A prediction module, configured to input the thick-layer CT image into a trained deep convolutional neural network model corresponding to the predetermined size to obtain a thin-layer CT prediction image; A post-processing module, configured to perform post-processing on the thin-layer CT prediction image to obtain a final thin-layer CT image; The patient's target part includes the head and neck, pelvis, and chest and abdomen, where: When the patient's target part is the head and neck or pelvis, the performing size adjustment and slicing processing on the thick-layer CT image according to the type of the patient's target part to obtain a thick-layer CT image with a predetermined size includes: Adjusting the size of the cropped thick-layer CT image to a first size; Performing slicing processing on the thick-layer CT image after size adjustment to obtain a first set of thick-layer CT images; When the patient's target part is the chest and abdomen, the performing size adjustment and slicing processing on the thick-layer CT image according to the type of the patient's target part to obtain a thick-layer CT image with a predetermined size includes: Adjust the size of the cropped thick-layer CT image to a first size; Obtain a local image of the spinal region with a second size from the cropped thick-layer CT image; Perform slicing processing on the thick-layer CT image whose size has been adjusted and the obtained local image respectively to obtain a second thick-layer CT image set and a third thick-layer CT image set.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the CT image processing method based on a deep convolutional neural network according to any one of claims 1 to 7.

Images