Image reconstruction model generation and image reconstruction method, device, equipment and medium

Abstract

The embodiment of the present invention discloses an image reconstruction model generation and image reconstruction method, device, equipment and medium, wherein the image reconstruction model generation method includes: obtaining a preliminary magnetic resonance reconstruction image, and reducing the resolution of the preliminary magnetic resonance reconstruction image according to a preset image resolution enhancement multiple to obtain a low-resolution image; performing image interpolation processing on the low-resolution image, and the multiple of the image interpolation is the preset image resolution enhancement multiple; using the interpolated image as input data for model training, using the preliminary magnetic resonance reconstruction image as label data, and training the high-resolution image reconstruction model, and when the loss function of the high-resolution image reconstruction model converges to a preset value, generating a target high-resolution image reconstruction model. The technical solution of this embodiment achieves the elimination of hidden dangers such as poor learning effect caused by data deviation, and can improve the resolution of the reconstructed image while shortening the imaging time, and has a better image effect.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of medical image processing, and in particular to an image reconstruction model generation and image reconstruction method, device, equipment and medium. Background Art

[0002] Magnetic Resonance Imaging (MRI) technology is widely used in clinical diagnosis and medical research due to its non-invasive, non-radiation, good soft tissue contrast and arbitrary layer imaging. At present, cardiac magnetic resonance movies have been considered the gold standard for imaging to evaluate cardiac function. However, high-resolution image acquisition in clinical practice often requires a long acquisition time. At the same time, due to the influence of patient tolerance and respiratory movement, it is often difficult to obtain high-resolution cardiac magnetic resonance images in actual clinical processes.

[0003] At present, the methods for obtaining high-resolution magnetic resonance cardiac movie images mainly include interpolation-based image reconstruction methods, traditional machine learning-based image reconstruction methods, and deep learning-based image reconstruction methods. Among them, the method based on deep learning network has good processing capabilities for both linear and nonlinear methods, and can reconstruct magnetic resonance images with higher image quality.

[0004] However, the existing methods for super-resolution image reconstruction based on deep learning have not solved the problem of data dependence, and still require a large number of paired low-resolution images and high-resolution images for learning. However, from the perspective of practical operation and patient ethics, as well as the particularity of medical images, paired high-resolution and low-resolution movie images are difficult to obtain. At the same time, for movie imaging of different resolutions, different field strengths and different phases, standardized registration is required, and this process is often inaccurate due to the influence of machine and geometric deformation. This makes the existing super-resolution methods based on deep learning vulnerable to data bias, increasing the difficulty of their large-scale clinical promotion. Summary of the invention

[0005] The embodiments of the present invention provide an image reconstruction model generation and image reconstruction method, device, equipment and medium, so as to shorten the imaging time and reduce the complexity of the image reconstruction network, while improving the reconstructed image resolution and achieving better image effects.

[0006] In a first aspect, an embodiment of the present invention provides a method for generating an image reconstruction model, the method comprising:

[0007] Acquire a preliminary magnetic resonance reconstruction image, and reduce the resolution of the preliminary magnetic resonance reconstruction image according to a preset image resolution enhancement factor to obtain a corresponding low-resolution image;

[0008] Performing image interpolation processing on the low-resolution image, wherein the multiple of the image interpolation is the preset image resolution enhancement multiple;

[0009] The image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstructed image corresponding to the low-resolution image is used as label data to train the high-resolution image reconstruction model. When the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.

[0010] Optionally, obtaining a preliminary magnetic resonance reconstruction image, and reducing the resolution of the preliminary magnetic resonance reconstruction image according to a preset image resolution enhancement factor to obtain a corresponding low-resolution image, including:

[0011] Inputting the preliminary magnetic resonance reconstruction image into a generator of a preset generative adversarial network, and having the generator perform convolution and downsampling processing on the preliminary magnetic resonance reconstruction image to obtain a constructed low-resolution image whose resolution is reduced by a multiple of the preset image resolution increase;

[0012] Extracting a discriminant image matching the constructed low-resolution image from the preliminary magnetic resonance reconstruction image, and inputting the discriminant image and the constructed low-resolution image into a discriminator of the preset generative adversarial network to train the preset generative adversarial network;

[0013] All convolution layer parameters of the generator in the trained preset generative adversarial network are convolved layer by layer to obtain the magnetic resonance image degradation kernel function;

[0014] The preliminary magnetic resonance reconstruction image is convolved with the magnetic resonance image degradation kernel function to obtain a corresponding low-resolution image.

[0015] Preferably, the generator of the preset generative adversarial network includes six convolutional layers, and the convolution step size of the last layer of the six convolutional layers is the preset image resolution enhancement multiple; the discriminator of the preset generative adversarial network includes seven convolutional layers.

[0016] Optionally, performing image interpolation processing on the low-resolution image includes:

[0017] The low-resolution image is subjected to bicubic interpolation processing to increase the preset image resolution multiple.

[0018] Optionally, before training the high-resolution image reconstruction model, the method further includes:

[0019] The image subjected to image interpolation processing and the preliminary magnetic resonance reconstruction image are synchronously rotated or mirrored, and the image pair obtained after the rotation or mirroring operation is used as new high-resolution image reconstruction model training sample data.

[0020] Optionally, during the training process of the high-resolution image reconstruction model, a residual learning method is adopted to add the residual of the input data after passing through the convolution layer to the input data itself, and then calculate the loss function between the input data and the label data.

[0021] In a second aspect, an embodiment of the present invention further provides an image reconstruction method, the method comprising:

[0022] Acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution enhancement multiple;

[0023] The preliminary magnetic resonance reconstructed image that has undergone image interpolation processing is input into a target image reconstruction model obtained by the image reconstruction model generation method described in any embodiment, which increases the resolution of the magnetic resonance image by the preset resolution increase multiple, to obtain a target magnetic resonance reconstructed image.

[0024] In a third aspect, an embodiment of the present invention further provides an image reconstruction model generation device, the device comprising:

[0025] An image degradation module is used to obtain a preliminary magnetic resonance reconstruction image, and reduce the resolution of the preliminary magnetic resonance reconstruction image according to a preset image resolution enhancement factor to obtain a corresponding low-resolution image;

[0026] An image interpolation module, used for performing image interpolation processing on the low-resolution image, wherein a multiple of the image interpolation is a multiple of the preset image resolution enhancement;

[0027] The model training module is used to use the image that has undergone image interpolation processing as input data for model training, and the preliminary magnetic resonance reconstructed image corresponding to the low-resolution image as label data, to train the high-resolution image reconstruction model, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.

[0028] Optionally, the image degradation module is specifically used for:

[0029] Inputting the preliminary magnetic resonance reconstruction image into a generator of a preset generative adversarial network, and having the generator perform convolution and downsampling processing on the preliminary magnetic resonance reconstruction image to obtain a constructed low-resolution image whose resolution is reduced by a multiple of the preset image resolution increase;

[0030] Extracting a discriminant image matching the constructed low-resolution image from the preliminary magnetic resonance reconstruction image, and inputting the discriminant image and the constructed low-resolution image into a discriminator of the preset generative adversarial network to train the preset generative adversarial network;

[0031] All convolution layer parameters of the generator in the trained preset generative adversarial network are convolved layer by layer to obtain the magnetic resonance image degradation kernel function;

[0032] The preliminary magnetic resonance reconstruction image is convolved with the magnetic resonance image degradation kernel function to obtain a corresponding low-resolution image.

[0033] Preferably, the generator of the preset generative adversarial network includes six convolutional layers, and the convolution step size of the last layer of the six convolutional layers is the preset image resolution enhancement multiple; the discriminator of the preset generative adversarial network includes seven convolutional layers.

[0034] Optionally, the image interpolation module is specifically used for:

[0035] The low-resolution image is subjected to bicubic interpolation processing to increase the preset image resolution multiple.

[0036] Optionally, the image reconstruction model generating device also includes a training sample enhancement module, which is used to synchronously rotate or mirror the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstructed image before training the high-resolution image reconstruction model, and use the image pair obtained after the rotation or mirroring operation as new high-resolution image reconstruction model training sample data.

[0037] Optionally, the model training module is also used to, during the training process of the high-resolution image reconstruction model, adopt a residual learning method to add the residual of the input data after passing through the convolution layer to the input data itself, and then calculate the loss function between the input data and the label data.

[0038] In a fourth aspect, an embodiment of the present invention further provides an image reconstruction device, the device comprising:

[0039] An image preprocessing module, used for acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution enhancement multiple;

[0040] An image reconstruction module is used to input the preliminary magnetic resonance reconstruction image that has undergone image interpolation processing into a target image reconstruction model that increases the resolution of the magnetic resonance image by a preset resolution increase multiple obtained by the image reconstruction model generation method described in any embodiment, to obtain a target magnetic resonance reconstructed image.

[0041] In a fifth aspect, an embodiment of the present invention further provides a computer device, the computer device comprising:

[0042] one or more processors;

[0043] A memory for storing one or more programs;

[0044] When the one or more programs are executed by the one or more processors, the one or more processors implement the image reconstruction model generation method or image reconstruction method provided by any embodiment of the present invention.

[0045] In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an image reconstruction model generation method or an image reconstruction method as provided in any embodiment of the present invention.

[0046] The embodiments of the above invention have the following advantages or beneficial effects:

[0047] In the embodiment of the present invention, a preliminary magnetic resonance reconstruction image is obtained, and the resolution of the preliminary magnetic resonance reconstruction image is reduced according to a preset image resolution enhancement multiple to obtain a corresponding low-resolution image; the low-resolution image is subjected to image interpolation processing, wherein the multiple of the image interpolation is the preset image resolution enhancement multiple; the image subjected to image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data to train the high-resolution image reconstruction model, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated. The problem of the dependence of magnetic resonance image reconstruction model training on data in the prior art is solved, and the training data and label data are obtained by degrading the magnetic resonance image itself currently obtained, so that the network training of the image reconstruction model can be completed, and the training of the neural network without the need to collect a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is realized, and learning can be performed using the internal part of the image itself, without the need for a large amount of paired image data, and it is an unsupervised learning method with fast imaging speed, and at the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of a method for generating an image reconstruction model provided in Embodiment 1 of the present invention;

[0049] Figure 2 It is a schematic diagram of a generative adversarial network structure provided by Embodiment 1 of the present invention;

[0050] Figure 31 is a schematic diagram of the network structure of a discriminator in a generative adversarial network provided in Embodiment 1 of the present invention;

[0051] Figure 4 It is a schematic diagram of a generator network structure in a generative adversarial network provided in Embodiment 1 of the present invention;

[0052] Figure 5 It is a schematic diagram of a network training process of an image reconstruction model provided in Embodiment 1 of the present invention;

[0053] Figure 6 is a flow chart of an image reconstruction method provided by Embodiment 2 of the present invention;

[0054] Figure 7 is a schematic diagram of an image reconstruction process provided by Embodiment 2 of the present invention;

[0055] Figure 8 This is a comparison diagram of magnetic resonance image reconstruction effects using different image reconstruction methods provided by the second embodiment of the present invention;

[0056] Fig. 9 is a structural schematic diagram of an image reconstruction model generation device provided by Embodiment 3 of the present invention;

[0057] Fig.10 is a structural schematic diagram of an image reconstruction device provided by a fourth embodiment of the present invention;

[0058] Fig.11 It is a structural diagram of a computer device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0059] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0060] Embodiment 1

[0061] Figure 1 This is a flow chart of a method for generating an image reconstruction model provided in Embodiment 1 of the present invention. This embodiment is applicable to the case where low-resolution magnetic resonance images themselves are used to train image reconstruction models. The method can be executed by an image reconstruction model generation device, which can be implemented by software and / or hardware and integrated into an electronic device with application development functions.

[0062] like Figure 1 As shown, the image reconstruction model generation method includes the following steps:

[0063] S110, obtaining a preliminary magnetic resonance reconstruction image, and reducing the resolution of the preliminary magnetic resonance reconstruction image according to a preset image resolution enhancement factor to obtain a corresponding low-resolution image.

[0064] Among them, the preliminary magnetic resonance reconstruction image is a magnetic resonance image that can be scanned and obtained by the current magnetic resonance imaging equipment, that is, an image with a relatively low resolution and has not been reconstructed with improved resolution (super-resolution reconstruction). The preset image resolution improvement multiple is the multiple by which the resolution of the preliminary magnetic resonance reconstruction image is expected to be improved. For example, if the resolution of the preliminary magnetic resonance reconstruction image is 128*128, it is hoped that the preliminary magnetic resonance reconstruction image can be reconstructed to obtain a high-resolution image with a resolution of 512*512; then, the preset image resolution improvement multiple can be obtained by dividing (512*512) by (128*128), and the preset image resolution improvement multiple is 16.

[0065] In this embodiment, the resolution of the preliminary magnetic resonance reconstruction image is reduced according to the preset image resolution enhancement multiple to obtain a corresponding low-resolution image, with the purpose of using the preliminary magnetic resonance reconstruction image itself as a training parameter of the image reconstruction network, rather than matching the preliminary magnetic resonance reconstruction image with a corresponding high-resolution high-quality magnetic resonance reconstruction image. Therefore, there is no need to collect a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data to train the neural network, which can reduce the difficulty of obtaining training samples for the image reconstruction model.

[0066] Specifically, the image resolution may be reduced by using a downsampling method to downsample the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image.

[0067] In a preferred embodiment, the generative adversarial network can also be used to degrade the preliminary magnetic resonance reconstruction image to obtain a low-resolution image with a resolution reduced by a preset image resolution increase multiple. In the process of training the generative adversarial network, a kernel function can be determined for performing the same image degradation on one or more preliminary magnetic resonance reconstruction images. Specifically, first, the preliminary magnetic resonance reconstruction image is input into Figure 2In the preset generative adversarial network shown in FIG. 1 , the generator G first performs convolution and downsampling processing on the preliminary magnetic resonance reconstruction image to obtain a low-resolution image with a resolution reduced by a preset image resolution increase multiple. The obtained image can be regarded as a forged version of the preliminary magnetic resonance reconstruction image, image F; further, an image block with the same size and matching position as image F is cut out from the preliminary magnetic resonance reconstruction image as a real sample image T. After image T and image F are input to the discriminator D together, the discriminator analyzes the possibility of image F being a real image pixel by pixel based on image T. The goal of the generator is to make the output image F deceive the discriminator as much as possible. In the process of mutual confrontation between the generator and the discriminator, the parameters are continuously adjusted. When the discriminator cannot determine whether the output result F of the generator is real, the network training is completed. In the heat map output by the discriminator, image F and image T are compared pixel by pixel. The probability value of each two pixels compared to each other is the same pixel. When the probability values ​​in the heat map all meet the probability values ​​of positive judgment, it can be determined that the discriminator cannot determine whether the output result F of the generator is real. Furthermore, all convolution layer parameters of the generator in the preset generative adversarial network that has been trained are convolved layer by layer to obtain the magnetic resonance image degradation kernel function; thus, each preliminary magnetic resonance reconstruction image can be convolved with the magnetic resonance image degradation kernel function to obtain a corresponding low-resolution image. In this embodiment, the size and number of convolution kernels in the generator in the preset generative adversarial network can be changed and adjusted. The generator includes six convolution kernels, and the sizes of each convolution kernel are: 5x5, 3x3, 1x1, 1x1, 1x1 and 1x1. In a preferred embodiment, the discriminator structure in the preset generative adversarial network can be Figure 3 The structure shown in the figure consists of 7 convolutional layers, and the convolution kernel sizes of each convolutional layer are 7x7, 1x1, 1x1, 1x1, 1x1, 1x1, and 1x1 respectively. The network structure of the generator is as follows Figure 4 As shown in the figure, it consists of 6 convolutional layers, and the convolution kernel sizes are: 7x7, 5x5, 3x3, 1x1, 1x1, 1x1. Among them, the convolution step value of the last convolutional layer of the generator is the preset image resolution enhancement multiple to achieve the effect of downsampling the input image.

[0068] It should be noted that if the magnetic resonance imaging object is a heart, then the obtained preliminary magnetic resonance reconstruction image is a cardiac magnetic resonance movie image, which includes multiple frames of magnetic resonance images. When performing generative adversarial network training, any frame of the cardiac magnetic resonance movie image can be taken as an input image.

[0069] S120: Perform image interpolation processing on the low-resolution image, wherein a multiple of the image interpolation is the preset image resolution enhancement multiple.

[0070] In this step, the low-resolution image is interpolated, and the image interpolation multiple is the preset image resolution enhancement multiple. This is to make the size of the input image of the image reconstruction model consistent with the size of the output image, which can shorten the model training time and optimize the model training process.

[0071] Specifically, in this embodiment, the interpolation method may use a bicubic interpolation algorithm, because bicubic interpolation can retain more image details during the image enlargement process, and the enlarged image has an anti-aliasing function, and the enlarged image is more realistic than the source image.

[0072] In a preferred embodiment, the image processed by image interpolation and the corresponding preliminary magnetic resonance reconstruction image can also be rotated or mirrored synchronously, and the image pair obtained after the rotation or mirroring operation is used as the new high-resolution image reconstruction model training sample data. For example, the image pair is rotated by 0 degrees, 90 degrees, 180 degrees and 270 degrees, and then the mirror symmetry operation in the horizontal and vertical directions is performed on this basis, so that 8 sets of training data can be obtained, so that the training data is enhanced. Moreover, the phases of the image pairs that have been synchronously processed are consistent, and there is no need for registration, thereby eliminating the hidden dangers of poor learning effects caused by data deviation.

[0073] S130. Use the image that has undergone image interpolation processing as input data for model training, use the preliminary magnetic resonance reconstructed image corresponding to the low-resolution image as label data, train the high-resolution image reconstruction model, and when the loss function of the high-resolution image reconstruction model converges to a preset value, generate a target high-resolution image reconstruction model.

[0074] After the above steps, a sufficient number of model training sample data are constructed, and the model training process can be started. The image after image interpolation processing is used as the input data for model training, and the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as the label data. The preliminary magnetic resonance image to be acquired is used as the label data, and the low-resolution image whose resolution is reduced by a preset image resolution improvement multiple is used. Then, when the preliminary magnetic resonance reconstruction image is input into the trained image reconstruction model, the image reconstruction model can output a high-resolution image whose resolution is increased by a preset image resolution improvement multiple.

[0075] For details, please refer to the training process of the target image reconstruction model. Figure 5 The process shown. Figure 5In the process shown, the image reconstruction network is a fully convolutional network with a total of 8 convolutional layers, where the convolution kernel size of the first layer is 3x3, the number of channels is f, the convolution kernel size of the second to seventh layers is 3x3, the number of channels is 64, and the convolution kernel size of the last layer is 3x3, the number of channels is f. Where f represents the number of image frames input into the image reconstruction network at the same time. For one image, the value of f is 1. When the collected preliminary magnetic resonance reconstruction image is a cardiac movie image, the value of f is the number of image frames in the cardiac movie image. The number of convolutional layers of the image reconstruction network can also be changed to 6 or more, and the convolution kernel size is not limited to 3x3, and the parameters can be adjusted according to the calculation requirements.

[0076] Furthermore, Figure 5 The model training process in the above example adopts a preferred deep learning method, namely residual learning method, which adds the residual of the input model image data after passing through the convolution layer to the input image data itself, and then calculates the loss function between the label data. It should be noted that the loss function can be used not only Figure 5 The L1 loss function shown in , L2 loss, perceptual loss and other loss functions can also be used. Furthermore, the network learning process can be optimized by using optimization algorithms such as Adam optimization algorithm, stochastic gradient descent algorithm or AdaGrad.

[0077] It should be noted here that in the structural diagrams of each network, conv represents the convolution kernel in the convolutional neural network, and ".mat" is the abbreviation of the file format. In this embodiment, it represents the magnetic resonance image degradation kernel function obtained by convolving all the convolution layer parameters of the generator in the preset generative adversarial network layer by layer.

[0078] The technical solution of this embodiment is to obtain a corresponding low-resolution image by reducing the image resolution of the acquired preliminary magnetic resonance reconstruction image, and the resolution reduction multiple can be controlled; then, the low-resolution image is subjected to image interpolation processing, wherein the multiple of the image interpolation is the preset image resolution enhancement multiple; the image subjected to image interpolation processing is used as the input data for model training, and the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as the label data to train the high-resolution image reconstruction model, and when the loss function of the high-resolution image reconstruction model converges to the preset value, the target high-resolution image reconstruction model is generated. The problem of the dependence of the magnetic resonance image reconstruction model training on data in the prior art is solved, and the training data and label data are obtained by degrading the magnetic resonance image itself, so as to complete the network training of the image reconstruction model, and realize the training of the neural network without the need to collect a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data, and the learning can be carried out by using the internal image itself, without the need for a large amount of paired image data. It is an unsupervised learning method with fast imaging speed, and at the same time, it eliminates the hidden dangers of poor learning effect caused by data deviation.

[0079] Embodiment 2

[0080] Figure 6 This is a flowchart of an image reconstruction method provided in Embodiment 2 of the present invention. This embodiment can be applied to the case where a low-resolution medical image acquired is reconstructed to obtain a high-resolution image. The method can be executed by an image reconstruction device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0081] like Figure 6 As shown, the image reconstruction method includes the following steps:

[0082] S210, obtaining a preliminary magnetic resonance reconstruction image, and performing image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution enhancement factor.

[0083] The preliminary magnetic resonance reconstruction image is a low-resolution image that needs to have its resolution improved. According to the resolution improvement requirement, the preliminary magnetic resonance reconstruction image can be interpolated and reconstructed to obtain a preprocessed image. The multiple of the image interpolation is the preset resolution improvement multiple.

[0084] S220. Input the preliminary magnetic resonance reconstructed image that has undergone image interpolation processing into a target image reconstruction model that is obtained by the image reconstruction model generation method described in any embodiment and that increases the resolution of the magnetic resonance image by the preset resolution increase multiple, to obtain a target magnetic resonance reconstructed image.

[0085] The target image reconstruction model is an image reconstruction model trained by the image reconstruction model generation method in the above embodiment according to the preset resolution enhancement requirement. Figure 7 Schematic diagram shown.

[0086] Furthermore, Figure 8 The figure shows the reconstructed images obtained by reconstructing the same preliminary magnetic resonance reconstructed image using different graphic reconstruction methods, where (a) NN is the result obtained by the nearest neighbor interpolation algorithm, (b) Bicubic is the result obtained by the bicubic interpolation algorithm, (c) zero-padding is the result of transforming the image to the Fourier domain, padding the surroundings with zeros, and then inversely transforming it back to the image domain, and (d) SR (Super Resolution) is the result obtained by the image reconstruction method of this embodiment. Intuitively, the image reconstruction result obtained by the image reconstruction method of this embodiment is clearer.

[0087] The technical solution of this embodiment obtains a reconstructed high-resolution image by inputting a low-resolution image into a pre-trained image reconstruction model that can increase the image resolution by a preset multiple; it solves the problems of long image reconstruction time and slow imaging speed in the prior art, and realizes that a high-resolution image with clear edges can be obtained in a relatively short time through an image reconstruction network with a relatively simple structure without a lot of calculations.

[0088] Embodiment 3

[0089] Fig. 9 This is a schematic diagram of the structure of an image reconstruction model generating device provided in Embodiment 3 of the present invention. This embodiment is applicable to the case where low-resolution magnetic resonance images themselves are used for image reconstruction model training.

[0090] like Fig. 9 As shown, the image reconstruction model generation device includes an image degradation module 310, an image interpolation module 320 and a model training module 330.

[0091] Among them, the image degradation module 310 is used to obtain a preliminary magnetic resonance reconstructed image, and reduce the resolution of the preliminary magnetic resonance reconstructed image according to a preset image resolution enhancement factor to obtain a corresponding low-resolution image; the image interpolation module 320 is used to perform image interpolation processing on the low-resolution image, wherein the image interpolation factor is the preset image resolution enhancement factor; the model training module 330 is used to use the image after image interpolation processing as input data for model training, and use the preliminary magnetic resonance reconstructed image corresponding to the low-resolution image as label data to train the high-resolution image reconstruction model, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.

[0092] The technical solution of this embodiment is to obtain a preliminary magnetic resonance reconstruction image, and reduce the resolution of the preliminary magnetic resonance reconstruction image according to a preset image resolution enhancement multiple to obtain a corresponding low-resolution image; perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple; use the image after image interpolation processing as input data for model training, use the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and train the high-resolution image reconstruction model, and when the loss function of the high-resolution image reconstruction model converges to a preset value, generate a target high-resolution image reconstruction model. The problem of data dependence in the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the magnetic resonance image itself currently obtained, so as to complete the network training of the image reconstruction model, and realize that there is no need to collect a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data to train the neural network, and the learning can be carried out by using the internal image itself, without the need for a large amount of paired image data. It is an unsupervised learning method with fast imaging speed, and at the same time, it eliminates the hidden dangers of poor learning effect caused by data deviation.

[0093] Optionally, the image degradation module 310 is specifically used for:

[0094] Inputting the preliminary magnetic resonance reconstruction image into a generator of a preset generative adversarial network, and having the generator perform convolution and downsampling processing on the preliminary magnetic resonance reconstruction image to obtain a constructed low-resolution image whose resolution is reduced by a multiple of the preset image resolution increase;

[0095] Extracting a discriminant image matching the constructed low-resolution image from the preliminary magnetic resonance reconstruction image, and inputting the discriminant image and the constructed low-resolution image into a discriminator of the preset generative adversarial network to train the preset generative adversarial network;

[0096] All convolution layer parameters of the generator in the trained preset generative adversarial network are convolved layer by layer to obtain the magnetic resonance image degradation kernel function;

[0097] The preliminary magnetic resonance reconstruction image is convolved with the magnetic resonance image degradation kernel function to obtain a corresponding low-resolution image.

[0098] Preferably, the generator of the preset generative adversarial network includes six convolutional layers, and the convolution step size of the last layer of the six convolutional layers is the preset image resolution enhancement multiple; the discriminator of the preset generative adversarial network includes seven convolutional layers.

[0099] Optionally, the image interpolation module 320 is specifically used for:

[0100] The low-resolution image is subjected to bicubic interpolation processing to increase the preset image resolution multiple.

[0101] Optionally, the image reconstruction model generating device also includes a training sample enhancement module, which is used to synchronously rotate or mirror the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstructed image before training the high-resolution image reconstruction model, and use the image pair obtained after the rotation or mirroring operation as new high-resolution image reconstruction model training sample data.

[0102] Optionally, the model training module 330 is also used to, during the training process of the high-resolution image reconstruction model, adopt a residual learning method to add the residual of the input data after passing through the convolution layer to the input data itself, and then calculate the loss function between the input data and the label data.

[0103] The image reconstruction model generation device provided in the embodiment of the present invention can execute the image reconstruction model generation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0104] Embodiment 4

[0105] Fig.10 This is a schematic diagram of the structure of an image reconstruction device provided in Embodiment 4 of the present invention. This embodiment is applicable to the case where a collected low-resolution medical image is reconstructed to obtain a high-resolution image.

[0106] like Fig.10 As shown, the image reconstruction device includes an image preprocessing module 410 and an image reconstruction module 420 .

[0107] Among them, the image preprocessing module 410 is used to obtain a preliminary magnetic resonance reconstructed image and perform image interpolation processing on the preliminary magnetic resonance reconstructed image to increase the resolution by a preset multiple; the image reconstruction module 420 is used to input the preliminary magnetic resonance reconstructed image after the image interpolation processing into the target image reconstruction model obtained by the image reconstruction model generation method described in any embodiment to increase the resolution of the magnetic resonance image by the preset resolution increase multiple, so as to obtain the target magnetic resonance reconstructed image.

[0108] The technical solution of this embodiment obtains a reconstructed high-resolution image by inputting a low-resolution image into a pre-trained image reconstruction model that can increase the image resolution by a preset multiple; it solves the problems of long image reconstruction time and slow imaging speed in the prior art, and realizes that a high-resolution image with clear edges can be obtained in a relatively short time through an image reconstruction network with a relatively simple structure without a lot of calculations.

[0109] The image reconstruction device provided in the embodiment of the present invention can execute the image reconstruction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0110] Embodiment 5

[0111] Fig.11 A schematic diagram of the structure of a computer device provided in Embodiment 5 of the present invention. Fig.11 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Fig.11 The computer device 12 shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capabilities connected to the magnetic resonance imaging device, such as an intelligent controller, a server, a mobile phone, and other terminal devices.

[0112] like Fig.11 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0113] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0114] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0115] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Fig.11 not shown, usually called a "hard drive"). Although Fig.11 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0116] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0117] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer device 12 via the bus 18. It should be understood that although Fig.11Not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0118] The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, implementing the steps of an image reconstruction model generation method provided in the embodiment of the present invention, the method comprising:

[0119] Acquire a preliminary magnetic resonance reconstruction image, and reduce the resolution of the preliminary magnetic resonance reconstruction image according to a preset image resolution enhancement factor to obtain a corresponding low-resolution image;

[0120] Performing image interpolation processing on the low-resolution image, wherein the multiple of the image interpolation is the preset image resolution enhancement multiple;

[0121] The image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstructed image corresponding to the low-resolution image is used as label data to train the high-resolution image reconstruction model. When the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.

[0122] Alternatively, an image reconstruction method provided by an embodiment of the present invention may be implemented, the method comprising:

[0123] Acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution enhancement multiple;

[0124] The preliminary magnetic resonance reconstructed image that has undergone image interpolation processing is input into a target image reconstruction model obtained by the image reconstruction model generation method described in any embodiment, which increases the resolution of the magnetic resonance image by the preset resolution increase multiple, to obtain a target magnetic resonance reconstructed image.

[0125] Embodiment 6

[0126] This sixth embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method for generating an image reconstruction model provided in any embodiment of the present invention is implemented, including:

[0127] Acquire a preliminary magnetic resonance reconstruction image, and reduce the resolution of the preliminary magnetic resonance reconstruction image according to a preset image resolution enhancement factor to obtain a corresponding low-resolution image;

[0128] Performing image interpolation processing on the low-resolution image, wherein the multiple of the image interpolation is the preset image resolution enhancement multiple;

[0129] The image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstructed image corresponding to the low-resolution image is used as label data to train the high-resolution image reconstruction model. When the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.

[0130] Alternatively, an image reconstruction method provided by an embodiment of the present invention may be implemented, the method comprising:

[0131] Acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution enhancement multiple;

[0132] The preliminary magnetic resonance reconstructed image that has undergone image interpolation processing is input into a target image reconstruction model obtained by the image reconstruction model generation method described in any embodiment, which increases the resolution of the magnetic resonance image by the preset resolution increase multiple, to obtain a target magnetic resonance reconstructed image.

[0133] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0134] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0135] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0136] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0137] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0138] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for generating an image reconstruction model, characterized in that: include: Acquire a preliminary magnetic resonance reconstruction image, input the preliminary magnetic resonance reconstruction image into a generator of a preset generative adversarial network, and have the generator perform convolution and downsampling processing on the preliminary magnetic resonance reconstruction image to obtain a constructed low-resolution image whose resolution is reduced by a multiple of the preset image resolution increase; Extracting a discriminant image matching the constructed low-resolution image from the preliminary magnetic resonance reconstruction image, and inputting the discriminant image and the constructed low-resolution image into a discriminator of the preset generative adversarial network to train the preset generative adversarial network; All convolution layer parameters of the generator in the trained preset generative adversarial network are convolved layer by layer to obtain the magnetic resonance image degradation kernel function; Convolving the preliminary magnetic resonance reconstruction image with the magnetic resonance image degradation kernel function to obtain a corresponding low-resolution image; Performing image interpolation processing on the low-resolution image, wherein the multiple of the image interpolation is the preset image resolution enhancement multiple; The image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstructed image corresponding to the low-resolution image is used as label data to train the high-resolution image reconstruction model. When the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.

2. The method according to claim 1, characterized in that The generator of the preset generative adversarial network includes six convolutional layers, and the convolution step size of the last layer of the six convolutional layers is the preset image resolution enhancement multiple; the discriminator of the preset generative adversarial network includes seven convolutional layers.

3. The method according to any one of claims 1-2, characterized in that: The performing image interpolation processing on the low-resolution image comprises: The low-resolution image is subjected to bicubic interpolation processing to increase the preset image resolution multiple.

4. The method according to any one of claims 1-2, characterized in that: Before training the high-resolution image reconstruction model, the method further includes: The image subjected to image interpolation processing and the preliminary magnetic resonance reconstruction image are synchronously rotated or mirrored, and the image pair obtained after the rotation or mirroring operation is used as new high-resolution image reconstruction model training sample data.

5. The method according to any one of claims 1-2, characterized in that: During the training process of the high-resolution image reconstruction model, a residual learning method is adopted to add the residual of the input data after passing through the convolution layer to the input data itself, and then calculate the loss function between the input data and the label data.

6. An image reconstruction method, characterized in that: include: Acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution enhancement multiple; The preliminary magnetic resonance reconstructed image after image interpolation processing is input into the target image reconstruction model obtained by the image reconstruction model generation method described in any one of claims 1 to 5, which improves the resolution of the magnetic resonance image by the preset resolution improvement multiple, to obtain the target magnetic resonance reconstructed image.

7. An image reconstruction model generation device, characterized in that: include: An image degradation module is used to obtain a preliminary magnetic resonance reconstruction image, input the preliminary magnetic resonance reconstruction image into a generator of a preset generative adversarial network, and the generator performs convolution and downsampling processing on the preliminary magnetic resonance reconstruction image to obtain a constructed low-resolution image with a resolution reduced by a multiple of the preset image resolution increase; Extracting a discriminant image matching the constructed low-resolution image from the preliminary magnetic resonance reconstruction image, and inputting the discriminant image and the constructed low-resolution image into a discriminator of the preset generative adversarial network to train the preset generative adversarial network; Convolving all convolution layer parameters of the generator in the trained preset generative adversarial network layer by layer to obtain a magnetic resonance image degradation kernel function; convolving the preliminary magnetic resonance reconstruction image with the magnetic resonance image degradation kernel function to obtain a corresponding low-resolution image; An image interpolation module, used for performing image interpolation processing on the low-resolution image, wherein a multiple of the image interpolation is a multiple of the preset image resolution enhancement; The model training module is used to use the image that has undergone image interpolation processing as input data for model training, and the preliminary magnetic resonance reconstructed image corresponding to the low-resolution image as label data, to train the high-resolution image reconstruction model, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.

8. An image reconstruction device, characterized in that: include: An image preprocessing module, used for acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution enhancement multiple; An image reconstruction module is used to input the preliminary magnetic resonance reconstruction image that has undergone image interpolation processing into a target image reconstruction model that increases the resolution of the magnetic resonance image by a preset resolution increase multiple obtained by the image reconstruction model generation method described in any one of claims 1 to 5, so as to obtain a target magnetic resonance reconstructed image.

9. A computer device, characterized in that: The computer device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image reconstruction model generation method or the image reconstruction method as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image reconstruction model generation method or the image reconstruction method as described in any one of claims 1 to 6 is implemented.

Images