Tomographic image reconstruction method, device, readable storage medium and electronic device
Through the non-uniform Fourier transform reconstruction network based on deep learning, the local and global correlation of tomographic image data is used to solve the long scanning time and radiation problems in tomographic image reconstruction, and high-quality artifact-free reconstruction images are achieved.
Patent Information
- Application Number
- CN202210056030.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-18
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-01-18
AI Technical Summary
The existing tomographic image reconstruction technology has problems such as excessive scanning time and ionizing radiation, making it difficult to effectively utilize the global correlation of image data in the space-frequency domain for high-quality reconstruction.
The non-uniform Fourier transform reconstruction network is used based on deep learning, and data filling is performed through local paths and global paths, and the reconstructed tomographic images are generated, using the local and global correlation of data in the space-frequency domain to achieve pseudo-free High-quality reconstruction of shadow.
It improves the efficiency and quality of tomographic image reconstruction, can effectively reduce radiation dose, is suitable for a variety of tomography methods, and improves the contrast and fine structure of the reconstructed image.
Smart Images

Figure CN114494493B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of tomographic image reconstruction, and in particular to a tomographic image reconstruction method, device, readable storage medium and electronic device. Background Art
[0002] Tomographic image reconstruction technology mainly includes magnetic resonance imaging (MRI), computed tomography (CT) and positron emission tomography (PET) technology. Medical workers can diagnose various diseases of the subject based on the image of the subject obtained by the tomographic image reconstruction technology. However, the above-mentioned tomographic image reconstruction technology has technical problems such as long scanning time and ionizing radiation, which can easily cause discomfort or harm to the subject. Therefore, the art urgently needs a tomographic image reconstruction method that can accelerate the scanning process or reduce the radiation dose.
[0003] Some researchers have proposed reconstruction algorithms based on parallel imaging principles, compressed sensing theory (CS) and deep learning. These methods all assume that the collected original image data is locally correlated in the space-frequency domain, that is, there is local redundancy. These methods can use this local redundancy to reconstruct and restore images under reduced sampling conditions. However, the original image data also has global correlation in the space-frequency domain, and there is no method that can make good use of this global correlation to reconstruct tomographic images.
[0004] Based on the above discussion, it can be seen that this field is in urgent need of a reconstruction method based on deep learning to directly optimize the non-uniform Fourier transform, directly reconstruct different types of tomographic image data, and utilize the global correlation of the data in the space-frequency domain to generate high-quality artifact-free reconstructed images. Summary of the invention
[0005] The present invention provides a method, a device, a readable storage medium and an electronic device for reconstructing a tomographic image, which solve the technical problem in the prior art that it is difficult to well utilize global correlation to reconstruct a tomographic image.
[0006] In a first aspect, the present invention provides a tomographic image reconstruction method, comprising:
[0007] Acquiring image data to be reconstructed obtained by tomography;
[0008] Using the pre-trained deep learning-based non-uniform Fourier transform reconstruction network, the image data to be reconstructed is filled in the local and global pathways, and then fused to obtain the reconstructed tomographic image.
[0009] In some embodiments, the training process of the non-uniform Fourier transform reconstruction network based on deep learning includes:
[0010] Acquire simulated image data of different tomography modes;
[0011] Build a non-uniform Fourier transform reconstruction network based on deep learning;
[0012] Use the simulated image data to train the deep learning-based non-uniform Fourier transform reconstruction network;
[0013] Based on the real collected image data of different tomography methods, the trained deep learning-based non-uniform Fourier transform reconstruction network is fine-tuned to obtain the final deep learning-based non-uniform Fourier transform reconstruction network.
[0014] In some embodiments, obtaining simulated image data of different tomography modes includes:
[0015] Grayscale processing is performed on multiple natural images to obtain corresponding grayscale images;
[0016] Randomly generate downsampled acquisition trajectories;
[0017] Different tomographic scanning modes are simulated for the grayscale image and the acquisition trajectory to obtain simulated image data of different tomographic scanning modes, and random Gaussian distribution noise is added to the simulated image data.
[0018] In some embodiments, a non-uniform Fourier transform reconstruction network based on deep learning includes:
[0019] A data rearrangement and scale transformation module is used to convert the simulated image data of different tomography modes into the same target form and generate a corresponding acquisition position map to record the acquisition position information of the data, and perform nonlinear scale transformation on the simulated image data of the target form;
[0020] An adaptive interpolation module is used to perform data filling processing on the local channel and the global channel respectively for the data after nonlinear scale transformation;
[0021] The fusion module is used to fuse the data of the local pathway and the global pathway after data filling processing to obtain a reconstructed tomographic image.
[0022] In some embodiments, when converting simulated image data of different tomography methods into the same target format, the data rearrangement and scale conversion module is used to:
[0023] When the simulated image data is simulated image data of an MRI scanning method, the simulated image data is converted into a square shape of a regular size;
[0024] When the simulated image data is simulated image data of a CT scanning method or simulated image data of a PET scanning method, a one-dimensional inverse Fourier transform is performed on the sinusoidal graph of the simulated image data to obtain the same form as the k-space data acquired by radial MRI, and then the simulated image data is converted into a square shape of regular size.
[0025] In some embodiments, when performing nonlinear scale transformation on the simulated image data in the target form, the data rearrangement and scale transformation module is used to:
[0026] The target simulated image data is nonlinearly scaled using the following formula:
[0027]
[0028] x t =x t / max(x t )
[0029]
[0030] Among them, x in is the simulated image data in the target form, ⊙ is the corresponding element multiplication operation, Ω is the template operation for obtaining the position of elements greater than zero, max is the maximum value operation, p is the hyperparameter, x t The output data is the nonlinear scale transformed data.
[0031] In some embodiments, when the adaptive interpolation module performs data filling processing on the data after nonlinear scale transformation in the local path and the global path respectively, it is used to:
[0032] Performing an expansion operation on the data after the nonlinear scale transformation to obtain each data point and the neighborhood of each data point in the data after the nonlinear scale transformation;
[0033] Performing point-by-point multiplication operations on each of the expanded data points and an adaptive density compensation function, and performing a convolution operation, to obtain fourth data; wherein the adaptive density compensation function is obtained by processing the acquisition position map using a weight generation network;
[0034] Performing group convolution on the fourth data to obtain fifth data, wherein the number of groups is the total number of data points;
[0035] The fifth data and the fourth data are respectively processed by data padding in the local path and the global path.
[0036] In some embodiments, performing data padding processing on the fifth data and the fourth data in the local path and the global path respectively includes:
[0037] In the local path, the fifth data is unrolled and the data is merged, and the merged data is placed in a collection area close to the center using the first spatial transformation operation;
[0038] In the global path, the fourth data is mapped to a low-dimensional space through an encoder and globally transformed, the globally transformed data is mapped back to the original space through a decoder, and the second spatial transformation operation is used to place the data in a collection area close to the periphery.
[0039] In some embodiments, when fusing the data of the local pathway and the global pathway after the data filling process to obtain the reconstructed tomographic image, the fusion module is used to:
[0040] The data of the local pathway and the global pathway after data filling are fused, and then the reconstructed tomographic image is obtained through inverse scaling and inverse Fourier transformation.
[0041] In a second aspect, the present invention provides a tomographic image reconstruction device, comprising:
[0042] An acquisition module, used for acquiring image data to be reconstructed obtained by tomography;
[0043] The reconstruction module is used to use a pre-trained deep learning-based non-uniform Fourier transform reconstruction network to fill in the image data to be reconstructed in the local pathway and the global pathway, and fuse them to obtain a reconstructed tomographic image.
[0044] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the method of any embodiment of the first aspect when the computer program is executed by a processor.
[0045] In a fourth aspect, the present invention provides an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and when the processor executes the computer program, a method of any embodiment of the first aspect is implemented.
[0046] The present invention provides a tomographic image reconstruction method, device, readable storage medium and electronic device, which collect image data to be reconstructed obtained by tomographic scanning; use a pre-trained non-uniform Fourier transform reconstruction network based on deep learning to perform data filling processing on the image data to be reconstructed in local and global pathways, and fuse them to obtain a reconstructed tomographic image; thereby directly optimizing the non-uniform Fourier transform operation, using a deep learning method to adaptively learn the convolution kernel based on a large amount of data, and obtaining the local and global correlations in the spatial frequency domain (i.e., the acquisition domain), so as to better restore the data and obtain a high-quality artifact-free reconstructed image. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] The present invention will be described in more detail below based on embodiments and with reference to the accompanying drawings:
[0048] Figure 1 A schematic flow chart of a tomographic image reconstruction method provided by an embodiment of the present invention;
[0049] Figure 2 A structural block diagram of a tomographic image reconstruction device provided by an embodiment of the present invention;
[0050] Figure 3 A schematic diagram of a tomographic image reconstruction network provided by an embodiment of the present invention;
[0051] Figure 4 A schematic diagram of data rearrangement provided by an embodiment of the present invention.
[0052] In the drawings, the same reference numerals are used for the same components, and the drawings are not drawn to scale. DETAILED DESCRIPTION
[0053] In order to enable those skilled in the art to better understand the scheme of the present invention, and to fully understand and implement how the present invention applies technical means to solve technical problems and achieve the corresponding technical effects, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only an embodiment of a part of the present invention, not all of the embodiments. The embodiments of the present invention and the various features in the embodiments can be combined with each other without conflict, and the technical schemes formed are all within the scope of protection of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention.
[0054] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0055] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0056] Tomographic image reconstruction technology mainly includes magnetic resonance imaging (MRI), computed tomography (CT) and positron emission tomography (PET) technology. Medical workers can diagnose various diseases of the subject based on the image of the subject obtained by the tomographic image reconstruction technology. However, the above-mentioned tomographic image reconstruction technology has technical problems such as long scanning time and ionizing radiation, which can easily cause discomfort or harm to the subject. Therefore, the art urgently needs a tomographic image reconstruction method that can accelerate the scanning process or reduce the radiation dose.
[0057] In recent years, some technical solutions have studied methods such as downsampled reconstruction of MRI, low-angle reconstruction of CT, and low-dose reconstruction of PET, all of which can reduce discomfort caused to the subjects. As a general tomographic image reconstruction method, non-uniform Fourier transform (NUFFT) can directly reconstruct MRI, CT, and PET, but this method has the following shortcomings:
[0058] First, non-uniform Fourier transform has a better reconstruction effect on full-sampled data or data without large noise. If there is insufficient data and interpolation loss, the reconstructed image will have large downsampling artifacts.
[0059] Secondly, in the non-uniform Fourier transform, a small-range convolution kernel with fixed parameters is used, which can only capture local information in the space-frequency domain (k-space) and cannot collect and restore global information well, thus affecting the reconstruction quality of the algorithm;
[0060] In addition, in the non-uniform Fourier transform, it is also necessary to calculate the density compensation function to adjust the density of the acquisition points. The quality of the density compensation function design directly affects the quality of reconstruction.
[0061] In order to solve the above shortcomings, some researchers have proposed reconstruction algorithms based on parallel imaging principles, compressed sensing theory (CS) and deep learning. These methods all assume that the collected original image data is locally correlated in the space-frequency domain, that is, there is local redundancy. These methods can use this local redundancy to reconstruct and restore images under reduced sampling conditions. However, the original image data still has global correlation in the space-frequency domain, and there is no method that can make good use of this global correlation for reconstruction.
[0062] Based on the above discussion, it can be seen that this field is in urgent need of a reconstruction method based on deep learning to directly optimize the non-uniform Fourier transform, directly reconstruct different types of tomographic image data, and utilize the global correlation of data in the space-frequency domain to generate high-quality artifact-free reconstructed images.
[0063] Embodiment 1
[0064] Figure 1 A schematic diagram of a flow chart of a tomographic image reconstruction method provided by an embodiment of the present invention. Figure 1 As shown, a tomographic image reconstruction method comprises:
[0065] Step S100, collecting image data to be reconstructed obtained by tomography;
[0066] Step S200, using a pre-trained deep learning-based non-uniform Fourier transform reconstruction network, the image data to be reconstructed is processed in the local pathway and the global pathway for data filling, and the reconstructed tomographic image is obtained by fusion.
[0067] A tomographic image reconstruction method provided in this embodiment acquires image data to be reconstructed obtained by tomographic scanning; uses a pre-trained non-uniform Fourier transform reconstruction network based on deep learning to perform data filling processing on the image data to be reconstructed in local and global pathways, and fuses them to obtain a reconstructed tomographic image; thereby directly optimizing the non-uniform Fourier transform operation, using a deep learning method to adaptively learn the convolution kernel based on a large amount of data, and obtaining the local and global correlation in the spatial frequency domain (i.e., the acquisition domain), so as to better restore the data and obtain a high-quality artifact-free reconstructed image.
[0068] Embodiment 2
[0069] Based on the above embodiment, the training process of the non-uniform Fourier transform reconstruction network based on deep learning in S200 includes:
[0070] Step S211, obtaining simulation image data of different tomography modes;
[0071] Step S212, building a non-uniform Fourier transform reconstruction network based on deep learning;
[0072] Step S213, using the simulated image data to train the constructed deep learning-based non-uniform Fourier transform reconstruction network;
[0073] Step S214, fine-tuning the trained deep learning-based non-uniform Fourier transform reconstruction network based on image data of different tomography modes actually collected, to obtain the final deep learning-based non-uniform Fourier transform reconstruction network.
[0074] In some embodiments, obtaining simulation image data of different tomography modes in step S211 includes:
[0075] Step S221, grayscale processing is performed on multiple natural images to obtain corresponding grayscale images;
[0076] Step S222, randomly generating a down-sampled acquisition trajectory;
[0077] Step S223, simulating different tomography modes on the grayscale image and the acquisition trajectory to obtain simulated image data of different tomography modes, and adding random Gaussian distribution noise to the simulated image data.
[0078] The above-mentioned tomography forms include CT, PET and MRI. In the form of CT or PET, the corresponding acquisition trajectory can be a sinusoidal graph, and in the form of MRI, the corresponding acquisition trajectory includes radial, spiral and propeller shapes. Based on a large number of grayscale images and acquisition trajectories, simulation of different tomography modes can be realized, which can increase the generalization ability of the network and have a better reconstruction effect on data of different acquisition trajectories. By adding 10-50dB of random Gaussian distribution noise to increase the diversity of data, the simulated image data is closer to the real image data; wherein, the simulated image data includes: k-space data in MRI, sinusoidal graph data in CT and sinusoidal graph data in PET.
[0079] In one example, the data is rearranged as Figure 4 As shown, reference numeral 101 is a schematic diagram of three types of MRI acquisition trajectories (radial, spiral and propeller), reference numeral 102 is a different acquisition trajectory, and different acquisition trajectories can be regarded as adjacent lines continuously acquired in space, which can be arranged closely to generate regular rectangular two-dimensional data; reference numeral 103 is the data actually acquired after rearrangement; reference numeral 104 is a position map corresponding to the data actually acquired.
[0080] In some cases, MRI data is used as the first data 101, and it is transformed into a square shape of regular size to obtain the second data 102. Similarly, the acquisition position map can also be transformed into the same shape; for CT data or PET data, the sinusoidal graph of the data is subjected to a one-dimensional inverse Fourier transform to obtain the same form as the k-space data obtained by radial MRI acquisition, and then transformed in the same way as the MRI data. This data rearrangement operation can standardize data of different modalities (MRI, CT, PET) to a unified format, and image reconstruction of a single network structure can be achieved without adjusting the network structure.
[0081] In this embodiment, natural images are used to generate simulated image data, and randomly generated downsampled acquisition trajectories and noise are used to increase the robustness of a deep learning-based non-uniform Fourier transform reconstruction network; a neural network structure is built, and data rearrangement and adaptive interpolation are used to reconstruct the data; the built network is trained on natural images, fine-tuned using real collected data, and applied to actual collected data for image reconstruction.
[0082] Embodiment 3
[0083] On the basis of the above embodiment, a non-uniform Fourier transform reconstruction network based on deep learning includes:
[0084] A data rearrangement and scale transformation module is used to convert the simulated image data of different tomography modes into the same target form and generate a corresponding acquisition position map to record the acquisition position information of the data, and perform nonlinear scale transformation on the simulated image data of the target form;
[0085] An adaptive interpolation module is used to perform data filling processing on the local channel and the global channel respectively for the data after nonlinear scale transformation;
[0086] The fusion module is used to fuse the data of the local pathway and the global pathway after data filling processing to obtain a reconstructed tomographic image.
[0087] In one example, build Figure 3 The deep learning-based non-uniform Fourier transform reconstruction network (DLNUFFT) is shown; first, the image data of different scanning modes are input as the first data 201 into the data rearrangement and scale transformation module 202 to be converted into image data of the same size, and a corresponding acquisition position map is generated to record the acquisition position information of the data; then, a nonlinear scale transformation is performed to increase its global correlation; the data is input into the adaptive interpolation module 203, and the local and global paths are used to fill and optimize the data, and finally the fusion module is used to obtain the restored data, and the Fourier transform is used to obtain the reconstructed image 204.
[0088] In some embodiments, when converting simulated image data of different tomography methods into the same target form, the data rearrangement and scale conversion module is used to:
[0089] When the simulated image data is simulated image data of an MRI scanning method, the simulated image data is converted into a square shape of a regular size;
[0090] When the simulated image data is simulated image data of CT scanning or simulated image data of PET scanning, a one-dimensional inverse Fourier transform is performed on the sinusoidal graph of the simulated image data to obtain the same form as the k-space data acquired by radial MRI, and then the simulated image data is converted into a square shape of regular size. In this way, data of different modalities (MRI, CT, PET) can be standardized to a unified format, and image reconstruction of a single network structure can be achieved without adjusting the network structure.
[0091] In some embodiments, the data rearrangement and scale transformation module is used to perform nonlinear scale transformation on the simulated image data of the target form:
[0092] The target simulated image data is nonlinearly scaled using the following formula:
[0093]
[0094] x t =x t / max(x t )
[0095]
[0096] Among them, x in is the simulated image data in the target form, ⊙ is the corresponding element multiplication operation, Ω is the template operation for obtaining the position of elements greater than zero, max is the maximum value operation, p is the hyperparameter, x t The output data is the nonlinear scale transformed data.
[0097] Still Figure 4 For example, a nonlinear scaling operation is performed on the second data 102 to obtain the third data 103, so as to enhance the global and local association between the data. The nonlinear scaling operation can be implemented by, but not limited to, the square root operation of the above expression. The input data x in the above formula is in Here is the second data 102, the value of the hyperparameter p can be 5, and the output data x in the above formula t This is the third data.
[0098] In some embodiments, the adaptive interpolation module is used to:
[0099] Performing an expansion operation on the data after the nonlinear scale transformation (third data) to obtain each data point and a neighborhood of each data point in the data after the nonlinear scale transformation;
[0100] Performing a point-by-point multiplication operation on each expanded data point and an adaptive density compensation function, and performing a convolution operation (such as a 1x1 convolution operation) to obtain fourth data; wherein the adaptive density compensation function is obtained by processing the acquisition position map using a weight generation network (Weigh Net);
[0101] Performing group convolution on the fourth data to obtain fifth data, wherein the number of groups is the total number of data points;
[0102] The fifth data and the fourth data are respectively processed by data padding in the local path and the global path.
[0103] Therefore, a better interpolation convolution and density compensation function can be provided compared with the non-uniform Fourier transform (NUFFT), thereby producing a better reconstructed image.
[0104] It should be understood that in actual application, using a pre-trained non-uniform Fourier transform reconstruction network based on deep learning, the image data to be reconstructed is processed in the local path and the global path, and the reconstructed tomographic image is obtained by fusing. It can be that the trained non-uniform Fourier transform reconstruction network based on deep learning is used to convert the image data to be reconstructed into a target form and generate a corresponding acquisition position map to record the acquisition position information of the data, and the target form of the simulated image data is subjected to nonlinear scale transformation; the data after the nonlinear scale transformation are respectively processed in the local path and the global path; the data of the local path and the global path after the data filling process are fused to obtain the reconstructed tomographic image. Wherein, when converting the image data to be reconstructed into the target form, different processing methods can be used according to the tomographic scanning method of the image data to be reconstructed. For the image data of the MRI scanning method, the image data to be reconstructed is converted into a square shape of regular size. For the image data of the CT scanning method or the image data of the PET scanning method, the sinusoidal diagram of the simulated image data is subjected to a one-dimensional inverse Fourier transform to obtain the same form as the k-space data obtained by the radial MRI acquisition, and then the simulated image data is converted into a square shape of regular size.
[0105] Embodiment 4
[0106] Based on the above embodiment, the fifth data and the fourth data are respectively subjected to data padding processing in the local path and the global path, including:
[0107] In the local path, the fifth data is unrolled and the data is merged, and the merged data is placed in a collection area close to the center using the first spatial transformation operation;
[0108] In the global path, the fourth data is mapped to a low-dimensional space through an encoder and globally transformed, the globally transformed data is mapped back to the original space through a decoder, and then the second space transformation operation is used to place the data in a collection area close to the periphery.
[0109] In some embodiments, when fusing the data of the local pathway and the global pathway after the data filling process to obtain the reconstructed tomographic image, the fusion module is used to:
[0110] The data of the local pathway and the global pathway after data filling are fused, and then the reconstructed tomographic image is obtained through inverse scaling and inverse Fourier transformation.
[0111] Therefore, not only the local correlation of the data in the space-frequency domain can be utilized, but also its global correlation can be utilized, so that the reconstructed image has better contrast information and fine structure, achieving better reconstruction effect.
[0112] In this embodiment, the two parts of data are fused in the fusion layer, and then subjected to inverse scaling transform and inverse Fourier transform to obtain a reconstructed image, which can not only utilize the local correlation of the data in the space-frequency domain, but also utilize its global correlation, so that the reconstructed image has better contrast information and fine structure, achieving a better reconstruction effect.
[0113] The following describes the different convolution modules involved in the deep learning-based non-uniform Fourier transform reconstruction network.
[0114] In the adaptive interpolation module, the weight generation network Weigh Net is composed of two groups of convolution-group normalization-leakage ReLU activation functions. The last layer is a repeated layer. The repeated layer will copy the data N times, N is the number of channels of the input data, the convolution kernel size is 3*3, and the number of channels is 8; the encoder Encoder is composed of two groups of convolution-group normalization-leakage ReLU activation functions, the convolution kernel size is 3*3, and the number of channels is 32; the decoder Decoder is composed of two groups of deconvolution-group normalization-leakage ReLU activation functions, the convolution kernel size is 3*3, and the number of channels is 32. The global transformation operation module GlobalTransform used in the global transformation operation includes three sets of leaky ReLU activation functions-matrix multiplication operations. The matrix size is the data size of the previous layer input; the first spatial transformation operation is a matrix multiplication operation. The matrix is a square matrix. The side length is the multiplication of the length and width of the data. The elements in the matrix are obtained from the position map, and the data can be linearly transformed to the collection area close to the center; the second spatial transformation operation is a matrix multiplication operation. The matrix is a square matrix. The side length is the multiplication of the length and width of the data. The elements in the matrix are obtained from the position map, and the data can be linearly transformed to the collection area close to the periphery.
[0115] The fusion module includes four groups of encoders and three groups of decoders. The convolution kernels of encoders are all 3*3, and the number of channels are 32, 64, 128, and 128 respectively. The convolution kernels of decoders are all 3*3, and the number of channels are 128, 64, and 32 respectively.
[0116] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:
[0117] The present invention combines the prior information of deep neural networks, reconstructs and optimizes the non-uniform Fourier transform reconstruction method, and can efficiently reconstruct a variety of high-quality artifact-free tomographic images;
[0118] Using global and local pathways, we can not only utilize the local correlation of data in the space-frequency domain, but also its global correlation, so that the reconstructed image has better contrast information and fine structure, achieving better reconstruction effect;
[0119] The method of combining natural image simulation data training with real data fine-tuning improves the robustness and usability of the method of the present invention;
[0120] The same network can be used to reconstruct different data and acquisition trajectories, which increases its versatility and has the potential for large-scale clinical promotion.
[0121] Embodiment 5
[0122] Figure 2 The structure block diagram of a tomographic image reconstruction device provided by an embodiment of the present invention. Based on the above embodiment, this embodiment provides a tomographic image reconstruction device, including:
[0123] An acquisition module, used for acquiring image data to be reconstructed obtained by tomography;
[0124] The reconstruction module is used to use a pre-trained deep learning-based non-uniform Fourier transform reconstruction network to fill in the image data to be reconstructed in the local pathway and the global pathway, and fuse them to obtain a reconstructed tomographic image.
[0125] The present embodiment provides a tomographic image reconstruction device, which acquires image data to be reconstructed by tomographic scanning; utilizes a pre-trained non-uniform Fourier transform reconstruction network based on deep learning to perform data filling processing on the image data to be reconstructed in local and global pathways, and fuses them to obtain a reconstructed tomographic image; thereby directly optimizing the non-uniform Fourier transform operation, using a deep learning method to adaptively learn the convolution kernel based on a large amount of data, and obtaining the local and global correlations in the spatial frequency domain (i.e., the acquisition domain), so as to better restore the data and obtain a high-quality artifact-free reconstructed image.
[0126] Embodiment 6
[0127] On the basis of the above embodiments, this embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method of the above embodiments is implemented.
[0128] The above-mentioned storage media can be flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, CD, server, App application store, etc.
[0129] For the content of the method, please refer to the aforementioned embodiment, which will not be repeated in this embodiment.
[0130] Embodiment 7
[0131] Based on the above embodiments, this embodiment provides an electronic device and a computer device, including a memory and a processor. The memory stores a computer program, and the processor implements the method of the above embodiments when executing the computer program.
[0132] The processor can be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components to implement the method in the above embodiment. For the content of the method, please refer to the above embodiment, which will not be repeated in this embodiment.
[0133] The memory can be implemented by any type of volatile or non-volatile storage device or a combination of them, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0134] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0135] It should be noted that, in the present invention, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element limited by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0136] Although the embodiments disclosed in the present invention are as above, the above contents are only embodiments adopted for facilitating the understanding of the present invention and are not intended to limit the present invention. Any technician in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present invention, but the patent protection scope of the present invention shall still be subject to the scope defined in the attached claims.
Claims
1. A tomographic image reconstruction method, characterized in that: include: Acquiring image data to be reconstructed obtained by tomography; Using a pre-trained non-uniform Fourier transform reconstruction network based on deep learning, the image data to be reconstructed is subjected to data filling processing in a local channel and a global channel, and the reconstructed tomographic image is obtained by fusing; Wherein, the non-uniform Fourier transform reconstruction network based on deep learning includes: A data rearrangement and scale transformation module is used to convert image data of different tomography modes into the same target form and generate a corresponding acquisition position map to record the acquisition position information of the data, and perform nonlinear scale transformation on the image data of the target form; An adaptive interpolation module is used to perform data filling processing on the local channel and the global channel respectively for the data after nonlinear scale transformation; A fusion module, used for fusing the data of the local pathway and the global pathway after the data filling process to obtain a reconstructed tomographic image; Wherein, when the adaptive interpolation module performs data filling processing on the data after nonlinear scale transformation in the local path and the global path respectively, it is used to: Performing an expansion operation on the data after the nonlinear scale transformation to obtain each data point and the neighborhood of each data point in the data after the nonlinear scale transformation; Performing point-by-point multiplication and convolution operations on each of the expanded data points and an adaptive density compensation function to obtain fourth data; wherein the adaptive density compensation function is obtained by processing the acquisition position map using a weight generation network; Performing group convolution on the fourth data to obtain fifth data, wherein the number of groups is the total number of data points; The fifth data and the fourth data are respectively processed by data padding in the local path and the global path; wherein the fifth data and the fourth data are respectively processed by data padding in the local path and the global path, comprising: In the local path, the fifth data is unrolled and the data is merged, and the merged data is placed in a collection area close to the center using the first spatial transformation operation; In the global path, the fourth data is mapped to a low-dimensional space through an encoder and globally transformed, the globally transformed data is mapped back to the original space through a decoder, and the second spatial transformation operation is used to place the data in a collection area close to the periphery.
2. The tomographic image reconstruction method according to claim 1, characterized in that: The training process of the non-uniform Fourier transform reconstruction network based on deep learning includes: Acquire simulated image data of different tomography modes; Build a non-uniform Fourier transform reconstruction network based on deep learning; Use the simulated image data to train the deep learning-based non-uniform Fourier transform reconstruction network; Based on the real collected image data of different tomography methods, the trained deep learning-based non-uniform Fourier transform reconstruction network is fine-tuned to obtain the final deep learning-based non-uniform Fourier transform reconstruction network.
3. The tomographic image reconstruction method according to claim 2, characterized in that: The obtaining of simulated image data of different tomography modes includes: Grayscale processing is performed on multiple natural images to obtain corresponding grayscale images; Randomly generate downsampled acquisition trajectories; The grayscale image and the acquisition trajectory are simulated in different tomographic scanning modes to obtain simulated image data in different tomographic scanning modes, and random Gaussian distribution noise is added to the simulated image data.
4. The tomographic image reconstruction method according to claim 1, characterized in that: The data rearrangement and scale conversion module is used to convert image data of different tomography modes into the same target form: When the image data is image data of an MRI scanning method, the image data is converted into a square shape of a regular size; When the image data is image data obtained by CT scanning or PET scanning, a one-dimensional inverse Fourier transform is performed on the sinusoidal graph of the image data to obtain the same form as the k-space data obtained by radial MRI acquisition, and then the image data is converted into a square shape of regular size.
5. The tomographic image reconstruction method according to claim 1, characterized in that: The data rearrangement and scale transformation module is used to perform nonlinear scale transformation on the target image data: The target image data is nonlinearly scaled using the following formula: x t =x t / max(x t ) Among them, x in is the target image data, ⊙ is the corresponding element multiplication operation, Ω is the template operation to obtain the position of the element greater than zero, max is the maximum value operation, p is the hyperparameter, x t The output data is the nonlinear scale transformed data.
6. The tomographic image reconstruction method according to claim 1, characterized in that: When the fusion module fuses the data of the local pathway and the global pathway after the data filling process to obtain the reconstructed tomographic image, it is used to: The data of the local pathway and the global pathway after data filling are fused, and then the reconstructed tomographic image is obtained through inverse scaling and inverse Fourier transformation.
7. A tomographic image reconstruction device based on the tomographic image reconstruction method according to any one of claims 1 to 6, characterized in that: include: An acquisition module, used for acquiring image data to be reconstructed obtained by tomography; The reconstruction module is used to utilize a pre-trained non-uniform Fourier transform reconstruction network based on deep learning to fill in the image data to be reconstructed in the local pathway and the global pathway, and fuse them to obtain a reconstructed tomographic image.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the tomographic image reconstruction method according to any one of claims 1 to 6 is implemented.
9. An electronic device, comprising a processor and a memory, characterized in that: The memory stores a computer program, and the processor implements the tomographic image reconstruction method according to any one of claims 1 to 6 when executing the computer program.
