CT image reconstruction method and device based on deep learning

Abstract

The application provides a CT image reconstruction method and device based on deep learning, and belongs to the technical field of artificial intelligence. The method comprises the following steps: inputting at least three continuous thick-layer CT images into a target model to obtain continuous multiple thin-layer CT images corresponding to a target thick-layer CT image; the target model comprises multiple parallel branches, and the multiple branches comprise a convolution branch and at least one transformer branch in sequence; the convolution branch is used for obtaining a target local feature; the transformer branch is used for obtaining a second fusion feature, performing global information extraction on the second fusion feature, and obtaining a target global feature. The CT image reconstruction method based on deep learning provided by the application can process at least three thick-layer CT images through the target model, reconstruct a thin-layer CT image, can obtain the thin-layer CT image without occupying a large storage space, reduces the acquisition cost of the thin-layer CT image, and improves the practicability of the thin-layer CT image in assisting doctors in diagnosis.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and apparatus for CT image reconstruction based on deep learning. Background Technology

[0002] High-resolution thin-slice CT images play a very good auxiliary role in both doctors' diagnosis and AI algorithms' lesion segmentation.

[0003] However, obtaining thin-slice CT images means that the patient spends more time in the CT room during the imaging process, resulting in increased radiation exposure. Furthermore, because thin-slice CT images have a larger data volume, they require more storage space during subsequent storage. Therefore, hospitals typically only acquire lower-resolution, thick-slice CT images and have no intention of storing thin-slice CT images.

[0004] In summary, developing a method to acquire thin-slice CT images without consuming excessive storage space is of great significance for reducing radiation exposure for patients and assisting doctors in diagnosis. Summary of the Invention

[0005] This application provides a deep learning-based CT image reconstruction method and apparatus to address the shortcomings of existing technologies where acquiring thin-slice CT images requires a large amount of storage space.

[0006] In a first aspect, embodiments of this application provide a deep learning-based CT image reconstruction method, comprising:

[0007] Input at least three consecutive thick-slice CT images into the target model to obtain multiple consecutive thin-slice CT images corresponding to the target thick-slice CT image output by the target model;

[0008] The target thick-slice CT image is the middle thick-slice CT image among the at least three consecutive thick-slice CT images; the resolution of the thin-slice CT image is greater than the resolution of the thick-slice CT image.

[0009] The target model includes multiple parallel branches, which in turn include a convolutional branch and at least one transformer branch.

[0010] The convolutional branch is used to obtain the first fused feature based on the output of each transformer branch, and to extract local information from the first fused feature to obtain the target local feature;

[0011] Each transformer branch is used to obtain a second fused feature based on the output of the convolutional branch and the outputs of other transformer branches, and to extract global information from the second fused feature to obtain the target global feature;

[0012] The output of the convolution branch and the output of the transformer are both determined based on the at least three consecutive thick-slice CT images;

[0013] The target model also includes an output module, which is connected to the multiple branches and is used to determine the consecutive multiple thin-slice CT images based on the target local features and the target global features of each transformer branch.

[0014] In one embodiment, in the first stage, the convolutional branch is used to extract local information from the at least three consecutive thick-slice CT images to obtain initial local features; the initial local features are the output of the convolutional branch in the first stage.

[0015] In each subsequent stage:

[0016] The target model adds a transformer branch that runs parallel to the convolution branch;

[0017] In the case where the previous stage includes multiple branches, the convolutional branch is used to fuse the outputs of each branch in the previous stage to obtain a first intermediate fused feature, and to extract local information from the first intermediate fused feature to obtain intermediate local features; the intermediate local features are the output of the convolutional branch in the current stage.

[0018] The transformer branch is used to fuse the outputs of each branch in the previous stage to obtain a second intermediate fused feature, and to extract global information based on the second intermediate fused feature to obtain an intermediate global feature; the intermediate global feature is the output of the transformer branch in the current stage.

[0019] Wherein, the target global feature is the intermediate global feature in the final stage; the target local feature is the intermediate local feature in the final stage.

[0020] In one embodiment, a deep learning-based CT image reconstruction method further includes:

[0021] The initial target model is trained based on at least three consecutive thick-slice CT image samples and their labels to obtain the target model;

[0022] The label is a series of thin-slice CT images corresponding to the at least three consecutive thick-slice CT image samples.

[0023] In one embodiment, the convolution branch includes multiple concatenated convolution blocks; the transformer branch includes multiple concatenated transformer blocks.

[0024] In one embodiment, the at least three consecutive thick-slice CT images comprise: at least three consecutive thick-slice CT images of the cross-section of the CT scan object.

[0025] In one embodiment, the CT scan interval between any two of the at least three consecutive thick-slice CT images is M, the number of the multiple consecutive thin-slice CT images is N, the CT scan interval between any two of the multiple consecutive thin-slice CT images is Y, and M = NY.

[0026] Secondly, embodiments of this application provide a deep learning-based CT image reconstruction apparatus, comprising:

[0027] The reconstruction module is used to input at least three consecutive thick-slice CT images into the target model to obtain multiple consecutive thin-slice CT images corresponding to the target thick-slice CT image output by the target model.

[0028] The target thick-slice CT image is the middle thick-slice CT image among the at least three consecutive thick-slice CT images; the resolution of the thin-slice CT image is greater than the resolution of the thick-slice CT image.

[0029] The target model includes multiple parallel branches, which in turn include a convolutional branch and at least one transformer branch.

[0030] The convolutional branch is used to obtain the first fused feature based on the output of each transformer branch, and to extract local information from the first fused feature to obtain the target local feature;

[0031] Each transformer branch is used to obtain a second fused feature based on the output of the convolutional branch and the outputs of other transformer branches, and to extract global information from the second fused feature to obtain the target global feature;

[0032] The output of the convolution branch and the output of the transformer are both determined based on the at least three consecutive thick-slice CT images;

[0033] The target model also includes an output module, which is connected to the multiple branches and is used to determine the consecutive multiple thin-slice CT images based on the target local features and the target global features of each transformer branch.

[0034] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the deep learning-based CT image reconstruction method of the first aspect.

[0035] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the deep learning-based CT image reconstruction method of the first aspect.

[0036] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the deep learning-based CT image reconstruction method of the first aspect.

[0037] The deep learning-based CT image reconstruction method provided in this application processes at least three thick-slice CT images through convolutional branches and at least one transformer branch in the target model to reconstruct clear and accurate thin-slice CT images. Only the thick-slice CT images need to be stored to obtain the thin-slice CT images, thereby enabling the acquisition of thin-slice CT images without occupying a large amount of storage space (no need to store thin-slice CT images). This reduces the cost of acquiring thin-slice CT images, helps reduce the radiation received by patients when taking thin-slice CT scans, and improves the practicality of thin-slice CT images in assisting doctors in diagnosis. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart of a deep learning-based CT image reconstruction method provided in an embodiment of this application;

[0040] Figure 2 This is a schematic diagram of the structure of the target model provided in the embodiments of this application;

[0041] Figure 3 This is a schematic diagram of one of the transformer branch structures provided in the embodiments of this application;

[0042] Figure 4 This is a schematic diagram of the structure of the deep learning-based CT image reconstruction device provided in the embodiments of this application;

[0043] Figure 5This is a schematic diagram of the physical structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] Figure 1 This is a flowchart of a deep learning-based CT image reconstruction method provided in an embodiment of this application. (Refer to...) Figure 1 This application provides a deep learning-based CT image reconstruction method, which may include:

[0046] Step S110: Input at least three consecutive thick-slice CT images into the target model to obtain multiple consecutive thin-slice CT images corresponding to the target thick-slice CT image output by the target model;

[0047] Among them, the target thick-slice CT image is the middle thick-slice CT image among at least three consecutive thick-slice CT images; the resolution of thin-slice CT images is greater than that of thick-slice CT images.

[0048] The target model consists of multiple parallel branches, each of which includes a convolutional branch and at least one transformer branch.

[0049] The convolutional branch is used to obtain the first fused feature based on the output of each transformer branch, and to extract local information from the first fused feature to obtain the target local feature;

[0050] Each transformer branch is used to obtain the second fused feature based on the output of the convolution branch and the output of other transformer branches, and to extract global information from the second fused feature to obtain the target global feature;

[0051] The output of the convolution branch and the output of the transformer are both determined based on at least three consecutive thick-slice CT images;

[0052] The target model also includes an output module, which is connected to multiple branches to determine multiple consecutive thin-slice CT images based on the target's local features and the target's global features in each transformer branch.

[0053] It should be noted that the target thick-slice CT image is any of the three consecutive thick-slice CT images except for the first and last images. For example, when the input thick-slice CT images are three consecutive thick-slice CT images, the target thick-slice CT image refers to the middle thick-slice CT image; when the input thick-slice CT images are five consecutive thick-slice CT images, the target thick-slice CT image can be any of the second, third, or fourth thick-slice CT images.

[0054] Since the target thick-slice CT image is located in the middle of at least three consecutive thick-slice CT images, the target model can make full use of the local and global features of other thick-slice CT images to provide a data basis for the target model to predict the final target thick-slice CT image from multiple consecutive thin-slice CT images, thereby significantly improving the accuracy of the target thick-slice CT image from multiple consecutive thin-slice CT images.

[0055] It should be noted that the target model includes convolutional branches arranged in sequence and at least one transformer branch. The convolutional branches are used to extract local features of the CT image, while the transformer branches are used to extract global features of the CT image.

[0056] Figure 2 This is a schematic diagram of the structure of the target model provided in the embodiments of this application. For example... Figure 2 As shown, a horizontally arranged branch consisting of several convolutional blocks or transformer blocks is a convolutional branch or a transformer branch.

[0057] When reconstructing CT images, the number of convolutional branches and transformer branches in the target model can be adjusted according to different needs. For example, when more local information needs to be extracted, four convolutional branches and two transformer branches can be set; when more local information needs to be extracted, two convolutional branches and five transformer branches can be set, and so on.

[0058] It should be noted that the number of convolutional branches and transformer branches can be adjusted, but the convolutional branches must be placed before the transformer branches. For example, in Figure 2 In the example shown, the first branch of the target model is a convolutional branch, and the second and third branches are transformer branches.

[0059] It should be noted that the convolution branch can extract local information from CT images specifically through convolutional blocks, while the transformer branch can extract global information from CT images specifically through transformer blocks. A convolutional block can include, for example, two convolutional operations, a batch normalization operation, and a ReLU activation function; the structure of a transformer block can be as follows: Figure 3 As shown.

[0060] After a CT image has been processed through any branch of the target model, the image size on that branch remains unchanged, thus allowing for more effective extraction of local or global information from the image.

[0061] It's important to note that during operation, the target model downsamples the image from the previous branch to obtain the image for the next branch. For example, if the image size in the first convolutional branch is 1024pt × 1024pt, the downsampled image for the second branch could be 512pt × 512pt, and the downsampled image for the third branch could be 256pt × 256pt, and so on. Correspondingly, when fusing the outputs of each branch, the target model upsamples the image output of the next branch to maintain consistency with the image size processed by the previous branch, thus facilitating feature fusion. For instance, the target model upsamples the image output of the second branch, enlarging its size from 512pt × 512pt to 1024pt × 1024pt to match the image size used in the first branch.

[0062] The output results of each branch can be merged using methods such as the conatenate function.

[0063] exist Figure 2 In the example shown, when the target model is running, the first branch (convolutional branch) can fuse the outputs of the second branch (transformer branch) and the third branch (transformer branch) to obtain the first fused feature, and further extract local information from the second fused feature to obtain the target local feature;

[0064] The second branch can fuse the outputs of the first branch and the third branch to obtain the second fused feature of the second branch, and then extract global information from the second fused feature to obtain the target global feature of the second branch.

[0065] The third branch can fuse the outputs of the first branch and the second branch to obtain the second fused feature of the third branch, and then extract global information from the second fused feature to obtain the target global feature of the third branch.

[0066] The output module can obtain multiple consecutive thin-slice CT images corresponding to the target thick-slice CT image based on the target local features, the target global features of the second branch, and the target global features of the third branch.

[0067] It is understandable that by extracting and fusing local and global features of thick-slice CT images through various branches, the local and global features in the thick-slice CT images can be more comprehensively expressed in the thin-slice CT images generated after processing by the target model, thereby improving the accuracy of the reconstructed thin-slice CT images.

[0068] The deep learning-based CT image reconstruction method provided in this application processes at least three thick-slice CT images through convolutional branches and at least one transformer branch in the target model to reconstruct clear and accurate thin-slice CT images. Only the thick-slice CT images need to be stored to obtain the thin-slice CT images, thereby enabling the acquisition of thin-slice CT images without occupying a large amount of storage space (no need to store thin-slice CT images). This reduces the cost of acquiring thin-slice CT images, helps reduce the radiation received by patients when taking thin-slice CT scans, and improves the practicality of thin-slice CT images in assisting doctors in diagnosis.

[0069] In one embodiment, the execution process of the target model includes multiple stages;

[0070] In the first stage, the convolutional branch is used to extract local information from at least three consecutive thick-slice CT images to obtain initial local features; the initial local features are the output of the convolutional branch in the first stage.

[0071] In each subsequent stage:

[0072] The target model adds a transformer branch that runs parallel to the convolution branch;

[0073] In the case where the previous stage includes multiple branches, the convolutional branch is used to fuse the outputs of each branch in the previous stage to obtain the first intermediate fused feature, and to extract local information from the first intermediate fused feature to obtain the intermediate local feature; the intermediate local feature is the output of the convolutional branch in the current stage.

[0074] The transformer branch is used to fuse the outputs of each branch in the previous stage to obtain the second intermediate fused feature, and to extract global information based on the second intermediate fused feature to obtain the intermediate global feature; the intermediate global feature is the output of the transformer branch in the current stage.

[0075] Among them, the target global features are the intermediate global features in the final stage; the target local features are the intermediate local features in the final stage.

[0076] exist Figure 2 In the example, in the first stage, the convolutional branch (first branch) can extract local information from at least three consecutive thick-slice CT images to obtain initial local features.

[0077] In the second stage, the target model can generate a second branch (transformer branch) that runs parallel to the convolutional branch.

[0078] The second branch can downsample the initial local features output by the convolutional branch in the first stage to obtain intermediate global features. It should be noted that since no transformer branch is generated in the first stage (i.e., the first stage only includes convolutional branches), the convolutional branch in the second stage does not need to perform feature fusion. Instead, it uses the initial local features as intermediate local features output by the convolutional branch in the second stage.

[0079] In the third stage, the target model generates another transformer branch (the third branch) that runs parallel to the first and second branches.

[0080] The first branch upsamples the intermediate global features output by the second branch in the second stage and then fuses them with the intermediate local features output by the first branch in the second stage to obtain the first intermediate fused feature of the first branch in the third stage. The image size corresponding to this first intermediate fused feature is the same as the image size corresponding to the intermediate local features of the first branch in the second stage. Next, the first branch extracts local information from this first intermediate fused feature to obtain intermediate local features.

[0081] The second branch can downsample the intermediate local features output by the first branch in the second stage and fuse them with the intermediate global features output by the second branch in the second stage to obtain the second intermediate fused feature of the second branch in the third stage. The size corresponding to this second intermediate fused feature is the same as the image size corresponding to the intermediate global features output by the second branch in the second stage. Then, the second branch will extract global information from this second intermediate fused feature to obtain the intermediate global feature.

[0082] The third branch can downsample and fuse the intermediate local features and intermediate global features output by the first branch in the second stage to obtain the second intermediate fused feature of the third branch in the third stage. Then, the third branch will extract global information from this second intermediate fused feature to obtain the intermediate global feature.

[0083] It should be noted that, due to Figure 2 The example target model's operation process consists of three stages; therefore, the intermediate local features and intermediate global features of the third stage are respectively the target local features and the target global features. Subsequently, the output module can output multiple consecutive thin-slice CT images based on these target local features and target global features.

[0084] The above explanation only uses the example of the target model's operation process including three stages. It is understood that the target model's operation process may include more stages, such as four, five, or six stages, etc., which are not limited in this invention.

[0085] Understandably, the operation process of a target model with more than three stages can refer to the operation process of the three stages mentioned above, and will not be repeated here.

[0086] Understandably, the number of stages in the execution process of the target model can be set according to the accuracy requirements of the target's global and local features, as well as the computing power of the processing device. For example, when the accuracy requirements of the target's global and local features are high, five stages can be set; when the accuracy requirements of the target's global and local features are low, three stages can be set.

[0087] The deep learning-based CT image reconstruction method provided in this application can effectively improve the feature representation and extraction of thick-slice CT images by enabling multiple branches of the target model to fully fuse and extract global and local information at different stages, thereby improving the accuracy of reconstructing thin-slice CT images from thick-slice CT images.

[0088] In one embodiment, the deep learning-based CT image reconstruction method may further include:

[0089] The initial target model is trained based on at least three consecutive thick-slice CT image samples and their labels to obtain the target model;

[0090] The label is a series of thin-slice CT images corresponding to at least three consecutive thick-slice CT image samples.

[0091] In one embodiment, an initial target model can be trained using a large number of samples and their corresponding labels. The samples consist of at least three consecutive thick-slice CT images, and the labels are the corresponding thin-slice CT images. Once the initial target model meets the accuracy requirements, the final target model can be obtained.

[0092] The deep learning-based CT image reconstruction method provided in this application improves the accuracy of reconstructing thin-slice CT images from thick-slice CT images by obtaining a target model through supervised training of an initial target model.

[0093] In one embodiment, a convolution branch may include multiple concatenated convolution blocks; a transformer branch may include multiple concatenated transformer blocks.

[0094] Understandably, since the convolutional branch and the transformer respectively use multiple concatenated convolutional blocks and multiple concatenated transformer blocks, the model structure can be simplified and the efficiency of image processing can be improved.

[0095] In one embodiment, at least three consecutive thick-slice CT images may include at least three consecutive thick-slice CT images of the cross-section of the CT scan object.

[0096] Understandably, cross-sectional continuity of a CT scan indicates strong correlation between CT images. Cross-sectional continuity allows for sufficiently small spacing between any two thick-slice CT images, resulting in high correlation between any two consecutive thick-slice CT images. This increases the accuracy of thin-slice CT image data predicted by the target model.

[0097] The deep learning-based CT image reconstruction method provided in this application improves the accuracy of the target model algorithm by continuously increasing the correlation between at least three thick-slice CT images of the cross-section of the CT scan object, thereby increasing the accuracy of the reconstructed thin-slice CT images and providing more accurate data support for lesion diagnosis.

[0098] In one embodiment, the CT scan interval between any two of at least three consecutive thick-slice CT images is M, the number of consecutive thin-slice CT images is N, the CT scan interval between any two of consecutive thin-slice CT images is Y, and M = NY.

[0099] It should be noted that the scanning interval between two adjacent images of at least three consecutive thick-slice CT images, the number of reconstructed thin-slice CT images, and the scanning interval between two adjacent images of consecutive reconstructed thin-slice CT images can have a certain quantitative relationship. For example, when three consecutive 5mm thick-slice CT images are input, the model will output five 1mm images after processing.

[0100] Understandably, because the CT scan interval between any two consecutive thin-slice CT images is equal, the features in the reconstructed thin-slice CT image are evenly distributed across the images. This avoids the poor continuity of feature data and low correlation between imaging results when the scan interval is not fixed, thus affecting the diagnostic results. For example, when five consecutive thin-slice CT images with a scan interval of 1 mm are constructed, the features in these five images are evenly distributed and the images have high correlation, making it easy to make a diagnosis based on the data presented in the preceding and following CT images. However, when five consecutive thin-slice CT images are constructed as two images with a scan interval of 1 mm and three images with a scan interval of 3 mm, the feature data between the images with a scan interval of 1 mm and the images with a scan interval of 3 mm jumps significantly, the correlation of the CT images is low, and it is not easy to make a diagnosis based on the data presented in the CT images.

[0101] The deep learning-based CT image reconstruction method provided in this application constructs thin-slice CT images with equal scanning intervals through a target model, so that the feature data on the thick-slice CT image can be evenly distributed in the reconstructed thin-slice CT, which increases the correlation of the reconstructed thin-slice CT image and further provides doctors with more accurate data for diagnosis.

[0102] Figure 4 This is a schematic diagram of the structure of the deep learning-based CT image reconstruction device provided in the embodiments of this application, with reference to... Figure 4 The device may include:

[0103] The reconstruction module 410 is used to input at least three consecutive thick-slice CT images into the target model to obtain multiple consecutive thin-slice CT images corresponding to the target thick-slice CT image output by the target model.

[0104] Among them, the target thick-slice CT image is the middle thick-slice CT image among at least three consecutive thick-slice CT images; the resolution of thin-slice CT images is greater than that of thick-slice CT images.

[0105] The target model consists of multiple parallel branches, each of which includes a convolutional branch and at least one transformer branch.

[0106] The convolutional branch is used to obtain the first fused feature based on the output of each transformer branch, and to extract local information from the first fused feature to obtain the target local feature;

[0107] Each transformer branch is used to obtain the second fused feature based on the output of the convolution branch and the output of other transformer branches, and to extract global information from the second fused feature to obtain the target global feature;

[0108] The output of the convolution branch and the output of the transformer are both determined based on at least three consecutive thick-slice CT images;

[0109] The target model also includes an output module, which is connected to multiple branches to determine multiple consecutive thin-slice CT images based on the target's local features and the target's global features in each transformer branch.

[0110] In the first stage, the convolutional branch is used to extract local information from at least three consecutive thick-slice CT images to obtain initial local features; the initial local features are the output of the convolutional branch in the first stage.

[0111] In each subsequent stage:

[0112] The target model adds a transformer branch that runs parallel to the convolution branch;

[0113] In the case where the previous stage includes multiple branches, the convolutional branch is used to fuse the outputs of each branch in the previous stage to obtain the first intermediate fused feature, and to extract local information from the first intermediate fused feature to obtain the intermediate local feature; the intermediate local feature is the output of the convolutional branch in the current stage.

[0114] The transformer branch is used to fuse the outputs of each branch in the previous stage to obtain the second intermediate fused feature, and to extract global information based on the second intermediate fused feature to obtain the intermediate global feature; the intermediate global feature is the output of the transformer branch in the current stage.

[0115] Among them, the target global features are the intermediate global features in the final stage; the target local features are the intermediate local features in the final stage.

[0116] In one embodiment, the apparatus further includes a training module for training an initial target model based on at least three consecutive thick-slice CT image samples and their labels to obtain a target model;

[0117] The label is a series of thin-slice CT images corresponding to at least three consecutive thick-slice CT image samples.

[0118] In one embodiment, a convolution branch may include multiple concatenated convolution blocks; a transformer branch may include multiple concatenated transformer blocks.

[0119] In one embodiment, at least three consecutive thick-slice CT images include: at least three consecutive thick-slice CT images of the cross-section of the CT scan object.

[0120] In one embodiment, the CT scan interval between any two of at least three consecutive thick-slice CT images is M, the number of consecutive thin-slice CT images is N, the CT scan interval between any two of consecutive thin-slice CT images is Y, and M = NY.

[0121] The deep learning-based CT image reconstruction device provided in this application process at least three thick-slice CT images through convolutional branches and at least one transformer branch in the target model to reconstruct clear and accurate thin-slice CT images. Only the thick-slice CT images need to be stored to obtain the thin-slice CT images, thereby enabling the acquisition of thin-slice CT images without occupying a large amount of storage space (no need to store thin-slice CT images). This reduces the cost of acquiring thin-slice CT images, helps reduce the radiation received by patients when taking thin-slice CT scans, and improves the practicality of thin-slice CT images in assisting doctors in diagnosis.

[0122] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute the deep learning-based CT reconstruction methods provided in the above embodiments, such as:

[0123] Input at least three consecutive thick-slice CT images into the target model to obtain multiple consecutive thin-slice CT images corresponding to the target thick-slice CT image output by the target model;

[0124] Among them, the target thick-slice CT image is the middle thick-slice CT image among at least three consecutive thick-slice CT images; the resolution of thin-slice CT images is greater than that of thick-slice CT images.

[0125] The target model consists of multiple parallel branches, each of which includes a convolutional branch and at least one transformer branch.

[0126] The convolutional branch is used to obtain the first fused feature based on the output of each transformer branch, and to extract local information from the first fused feature to obtain the target local feature;

[0127] Each transformer branch is used to obtain the second fused feature based on the output of the convolution branch and the output of other transformer branches, and to extract global information from the second fused feature to obtain the target global feature;

[0128] The output of the convolution branch and the output of the transformer are both determined based on at least three consecutive thick-slice CT images;

[0129] The target model also includes an output module, which is connected to multiple branches to determine multiple consecutive thin-slice CT images based on the target's local features and the target's global features in each transformer branch.

[0130] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0131] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the deep learning-based CT reconstruction method provided in the above embodiments, for example including:

[0132] Input at least three consecutive thick-slice CT images into the target model to obtain multiple consecutive thin-slice CT images corresponding to the target thick-slice CT image output by the target model;

[0133] Among them, the target thick-slice CT image is the middle thick-slice CT image among at least three consecutive thick-slice CT images; the resolution of thin-slice CT images is greater than that of thick-slice CT images.

[0134] The target model consists of multiple parallel branches, each of which includes a convolutional branch and at least one transformer branch.

[0135] The convolutional branch is used to obtain the first fused feature based on the output of each transformer branch, and to extract local information from the first fused feature to obtain the target local feature;

[0136] Each transformer branch is used to obtain the second fused feature based on the output of the convolution branch and the output of other transformer branches, and to extract global information from the second fused feature to obtain the target global feature;

[0137] The output of the convolution branch and the output of the transformer are both determined based on at least three consecutive thick-slice CT images;

[0138] The target model also includes an output module, which is connected to multiple branches to determine multiple consecutive thin-slice CT images based on the target's local features and the target's global features in each transformer branch.

[0139] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the deep learning-based CT image reconstruction methods provided in the above embodiments, including, for example:

[0140] Input at least three consecutive thick-slice CT images into the target model to obtain multiple consecutive thin-slice CT images corresponding to the target thick-slice CT image output by the target model;

[0141] Among them, the target thick-slice CT image is the middle thick-slice CT image among at least three consecutive thick-slice CT images; the resolution of thin-slice CT images is greater than that of thick-slice CT images.

[0142] The target model consists of multiple parallel branches, each of which includes a convolutional branch and at least one transformer branch.

[0143] The convolutional branch is used to obtain the first fused feature based on the output of each transformer branch, and to extract local information from the first fused feature to obtain the target local feature;

[0144] Each transformer branch is used to obtain the second fused feature based on the output of the convolution branch and the output of other transformer branches, and to extract global information from the second fused feature to obtain the target global feature;

[0145] The output of the convolution branch and the output of the transformer are both determined based on at least three consecutive thick-slice CT images;

[0146] The target model also includes an output module, which is connected to multiple branches to determine multiple consecutive thin-slice CT images based on the target's local features and the target's global features in each transformer branch.

[0147] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0148] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A deep learning-based CT image reconstruction method, characterized in that, include: Input at least three consecutive thick-slice CT images into the target model to obtain multiple consecutive thin-slice CT images corresponding to the target thick-slice CT image output by the target model; The target thick-slice CT image is the middle thick-slice CT image among the at least three consecutive thick-slice CT images; the resolution of the thin-slice CT image is greater than the resolution of the thick-slice CT image. The target model includes multiple parallel branches, which in turn include a convolutional branch and at least one transformer branch. The convolutional branch is used to obtain the first fused feature based on the output of each transformer branch, and to extract local information from the first fused feature to obtain the target local feature; Each transformer branch is used to obtain a second fused feature based on the output of the convolutional branch and the outputs of other transformer branches, and to extract global information from the second fused feature to obtain the target global feature; The output of the convolution branch and the output of the transformer branch are both determined based on the at least three consecutive thick-slice CT images; The target model also includes an output module, which is connected to the multiple branches and is used to determine the consecutive multiple thin-slice CT images based on the target local features and the target global features of each transformer branch. The operation of the target model includes multiple stages; In the first stage, the convolutional branch is used to extract local information from the at least three consecutive thick-slice CT images to obtain initial local features; the initial local features are the output of the convolutional branch in the first stage. In each subsequent stage: The target model adds a transformer branch that runs parallel to the convolution branch; In the case where the previous stage includes multiple branches, the convolutional branch is used to fuse the outputs of each branch in the previous stage to obtain a first intermediate fused feature, and to extract local information from the first intermediate fused feature to obtain intermediate local features; the intermediate local features are the output of the convolutional branch in the current stage. The transformer branch is used to fuse the outputs of each branch in the previous stage to obtain a second intermediate fused feature, and to extract global information based on the second intermediate fused feature to obtain an intermediate global feature; the intermediate global feature is the output of the transformer branch in the current stage. Wherein, the target global feature is the intermediate global feature in the final stage; the target local feature is the intermediate local feature in the final stage.

2. The CT image reconstruction method based on deep learning according to claim 1, characterized in that, The method further includes: The initial target model is trained based on at least three consecutive thick-slice CT image samples and their labels to obtain the target model; The label is a series of thin-slice CT images corresponding to the at least three consecutive thick-slice CT image samples.

3. The deep learning-based CT image reconstruction method according to claim 1, characterized in that, The convolution branch includes multiple concatenated convolution blocks; the transformer branch includes multiple concatenated transformer blocks.

4. The deep learning-based CT image reconstruction method according to any one of claims 1 to 3, characterized in that, The aforementioned at least three consecutive thick-slice CT images include: at least three consecutive thick-slice CT images of the cross-section of the CT scan object.

5. The deep learning-based CT image reconstruction method according to any one of claims 1 to 3, characterized in that, The CT scan interval between any two of the at least three consecutive thick-slice CT images is M, the number of the multiple consecutive thin-slice CT images is N, the CT scan interval between any two of the multiple consecutive thin-slice CT images is Y, and M=NY.

6. A CT image reconstruction device based on deep learning, characterized in that, include: The reconstruction module is used to input at least three consecutive thick-slice CT images into the target model to obtain multiple consecutive thin-slice CT images corresponding to the target thick-slice CT image output by the target model. The target thick-slice CT image is the middle thick-slice CT image among the at least three consecutive thick-slice CT images; the resolution of the thin-slice CT image is greater than the resolution of the thick-slice CT image. The target model includes multiple parallel branches, which in turn include a convolutional branch and at least one transformer branch. The convolutional branch is used to obtain the first fused feature based on the output of each transformer branch, and to extract local information from the first fused feature to obtain the target local feature; Each transformer branch is used to obtain a second fused feature based on the output of the convolutional branch and the outputs of other transformer branches, and to extract global information from the second fused feature to obtain the target global feature; The output of the convolution branch and the output of the transformer branch are both determined based on the at least three consecutive thick-slice CT images; The target model also includes an output module, which is connected to the multiple branches and is used to determine the consecutive multiple thin-slice CT images based on the target local features and the target global features of each transformer branch. The operation of the target model includes multiple stages; In the first stage, the convolutional branch is used to extract local information from the at least three consecutive thick-slice CT images to obtain initial local features; the initial local features are the output of the convolutional branch in the first stage. In each subsequent stage: The target model adds a transformer branch that runs parallel to the convolution branch; In the case where the previous stage includes multiple branches, the convolutional branch is used to fuse the outputs of each branch in the previous stage to obtain a first intermediate fused feature, and to extract local information from the first intermediate fused feature to obtain intermediate local features; the intermediate local features are the output of the convolutional branch in the current stage. The transformer branch is used to fuse the outputs of each branch in the previous stage to obtain a second intermediate fused feature, and to extract global information based on the second intermediate fused feature to obtain an intermediate global feature; the intermediate global feature is the output of the transformer branch in the current stage. Wherein, the target global feature is the intermediate global feature in the final stage; the target local feature is the intermediate local feature in the final stage.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the deep learning-based CT image reconstruction method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the deep learning-based CT image reconstruction method as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the deep learning-based CT image reconstruction method as described in any one of claims 1 to 5.

Images