Image reconstruction method, apparatus and electronic device

Abstract

The application provides an image reconstruction method and device and electronic equipment, wherein the image reconstruction method comprises: performing block processing on an acquired magnetic resonance image; performing processing on the image after block processing by using a blind restoration algorithm to obtain a plurality of restored image blocks; splicing the plurality of image blocks; and inputting the spliced image into a pre-trained reconstruction model to obtain a reconstructed image. The image is subjected to block prediction and blind restoration, and finally spliced, and the reconstructed model is inputted for reconstruction. The blind restoration can utilize estimation of a blur kernel to perform motion blur repair and high-resolution prediction, randomly predict a trajectory of a motion artifact of a medical image with spatially variant properties, prevent the motion blur from enhancing the artifact in the resolution reconstruction process, and achieve the purpose of reducing the enhancement of the motion artifact.

Description

Technical Field

[0001] The present invention relates to the technical field of magnetic resonance medical image processing, and in particular to an image reconstruction method, device and electronic equipment. Background Art

[0002] Scan time is a very important parameter in magnetic resonance imaging. Scans that take several minutes or even tens of minutes make it difficult for patients to remain still for too long, which can easily lead to motion artifacts. At the same time, the signal-to-noise ratio is also an important parameter in most scans, but improving the signal-to-noise ratio often requires increasing the layer thickness, which reduces the spatial resolution of the signal. In order to speed up the scan and improve the signal-to-noise ratio (SNR), the imaging time is usually shortened by increasing the layer thickness and increasing the sampling step during imaging. However, this often leads to lower image resolution and the appearance of volume effect.

[0003] Low spatial resolution has a significant negative impact on the observation of tissue details, such as the display of microvessels in MRA and signal registration in post-processing. Therefore, high-resolution images come at the expense of scanning time and SNR. In past studies, super-resolution technology has often been used to improve this problem. Super-resolution technology reconstructs higher-resolution (High-Resolution, HR) images from low-resolution (Low-Resolution, LR) images that are blurred and undersampled due to various hardware and scanning condition limitations. It uses a shorter time to collect lower-resolution images, and after training through a neural network, it gives a predicted output of a high-resolution image.

[0004] In 2014, Dong et al. first proposed the Super-Resolution Convolutional Neural Network (SRCNN), using the SRCNN model for super-resolution reconstruction of low-resolution images to high resolution. In 2016, Dong et al. improved the SRCNN network and proposed FSRCNN (Fast Super-Resolution Convolutional Neural Network). Compared to SRCNN, this network uses a final deconvolution layer to increase the size while changing the feature dimension, using smaller convolution kernels and more mapping layers. It can also share mapping layers, thereby speeding up training. In 2016, VDSR (Very Deep Convolutional Networks for Super-Resolution) was proposed. This network first used a residual method to solve the super-resolution problem. This method is based on the principle that the low-frequency information of the low-resolution and high-resolution images of the same image is similar. The residual information of the two is used as the learning target. The deeper the network, the better the model's expressiveness.

[0005] In recent years, most of the technologies related to super-resolution reconstruction of magnetic resonance imaging based on deep learning are based on SRCNN, FSRCNN, multi-scale fusion CNN, etc.

[0006] Super-resolution algorithms: Currently, the deep learning networks commonly used for super-resolution reconstruction are mainly based on SRCNN and FSRCNN. Due to their lack of depth, these networks have a small receptive field. Especially when performing large-scale transformations, they cannot obtain deep information from small image blocks, resulting in poor reconstruction quality. Furthermore, these networks converge slowly and can only achieve super-resolution reconstruction at a single scale.

[0007] At the same time, since current super-resolution technology is mostly aimed at natural images, it only uses deep learning networks to learn the features of low-resolution images. However, for magnetic resonance images, due to long-term scanning, patients may experience unconscious or conscious movements. In addition, due to hardware and other reasons, various types of blur and motion artifacts are more common. After super-resolution reconstruction, the image may be accompanied by the enhancement of various artifacts.

[0008] In terms of blind restoration: Previously, there were few cases of applying image restoration algorithms to magnetic resonance imaging. The goal of some methods of applying image restoration to magnetic resonance images was to predict the PSF for noise processing in magnetic resonance imaging. However, in recent years, blind restoration has been proven in other fields to be able to process multi-dimensional motion blur.

[0009] Although blind motion blur restoration methods are commonly used in the field of machine vision, they are not popular in the field of magnetic resonance imaging. The reason why motion blur in natural images and medical images captured by cameras is that the motion blur in some aspects is very different. The noise of the image captured by the camera and the original image depend on the exposure time and the operator's operating skills. For example, when people use a specific camera and a known image acquisition scene setting, its motion parameters are fixed, but the PSF of medical images is more difficult to predict. The acquisition of the blur kernel caused by the interaction of patient motion blur, hardware-induced noise and artifacts, and faster acquisition time needs to be regarded as a random process.

[0010] The impact of spatial variability in MRI: Super-resolution and image restoration algorithms are both global, but in MRI images, the blur kernel caused by motion and hardware is not global. Directly processing the image globally can speed up processing, reduce preprocessing steps, and reduce computational complexity, but it ignores the impact of the spatial variability of the PSF itself applied to the global image relative to the MRI image itself.

[0011] In summary, the existing technology has the following problems:

[0012] 1. Currently, the deep learning networks commonly used for super-resolution reconstruction are mainly based on SRCNN and FSRCNN. Due to their lack of depth, these networks have a small receptive field. Especially when performing large-scale transformations, they cannot obtain deep information from small image patches, resulting in poor reconstruction quality. Furthermore, these networks converge slowly and can only achieve super-resolution reconstruction at a single scale.

[0013] 2. The disadvantage of the currently commonly used super-resolution network is that it only uses low-resolution images as input, and the predicted high-resolution image output lacks a lot of necessary information, which makes it difficult to improve the blurred edges of the image. At the same time, due to the existence of motion artifacts in the magnetic resonance image itself, this method may have an amplifying effect on motion artifacts.

[0014] 3. There are difficulties in applying image restoration algorithms to magnetic resonance imaging in existing technologies. The reason why motion blur blind restoration methods are very common in the field of machine vision but have not been popularized in the field of magnetic resonance imaging is that the motion blur of natural images and medical images captured by cameras differs greatly in some aspects. That is, the noise of the image captured by the camera and the original image both depend on the exposure time and the operator's operating skills. For example, when people use a specific camera and a known image acquisition scene setting, their motion parameters are fixed, but the PSF of medical images is more difficult to predict.

[0015] 4. Direct global processing of the image can speed up the processing to a certain extent, reduce the preprocessing steps, and reduce the computational complexity. However, it ignores the effect of the PSF itself on the spatial variability of the magnetic resonance image itself. That is, in the magnetic resonance image, the blur kernel caused by motion and hardware is not global. Summary of the Invention

[0016] To solve the above problems, an object of the embodiments of the present invention is to provide an image reconstruction method, apparatus, and electronic device, which at least solve the problem of enhancing motion artifacts in the prior art.

[0017] In a first aspect, an embodiment of the present invention discloses an image reconstruction method, comprising:

[0018] Performing block processing on the acquired magnetic resonance images;

[0019] The image after block processing is processed using a blind restoration algorithm to obtain multiple restored image blocks;

[0020] splicing a plurality of the image blocks;

[0021] The spliced ​​image is input into the pre-trained reconstruction model to obtain the reconstructed image.

[0022] Optionally, the block-processed image is processed using a blind restoration algorithm to obtain a plurality of restored image blocks, including:

[0023] Perform image modeling and select corresponding parameters based on the collected signals, wherein the parameters are parameters of the model obtained by image modeling;

[0024] Predict motion blur kernel based on image modeling;

[0025] performing an error analysis on the motion blur kernel and creating a restoration-error pair model based on the error analysis;

[0026] Obtaining an accurate motion blur kernel based on the restoration-error pair model;

[0027] Non-blind image restoration is performed based on the precise motion blur kernel to obtain a restored image.

[0028] Optionally, the image modeling and corresponding parameter selection based on the collected signal, wherein the parameters are parameters of the model obtained by image modeling, include:

[0029] After signal acquisition at time t, the image acquired by modeling is:

[0030] z t =k(u t (x)+∈(x)),x∈X,

[0031] refers to the discrete grid of sampling, k is used to normalize the signal to a limited dynamic range u t (x) represents the time-varying and shift-varying Poisson distribution, and ∈(x) represents the time-invariant and shift-invariant Gaussian distribution.

[0032] Optionally, obtaining an accurate motion blur kernel based on the restoration-error pair model includes:

[0033] Choose a time from a limited set of times;

[0034] Obtaining a restoration-error plane based on the time and restoration-error pair model;

[0035] The predicted acquisition signal time is selected based on the restoration-error plane, and the optimal acquisition time solution is obtained through iteration;

[0036] An accurate motion blur kernel is obtained based on the optimal acquisition time solution.

[0037] Optionally, the restoration-error pair model is:

[0038]

[0039] y is the input image, λ and σ are the user input parameters of the two types of kernel distributions, r is the expected restoration error of the restoration-error pair model, θ1, θ2, ..., θ n is the descriptor of the motion blur kernel, is a statistical method, t is the signal acquisition time, h t is the motion blur kernel, H t is the set of models used to calculate the restoration error.

[0040] Optionally, performing non-blind image restoration based on the precise motion blur kernel to obtain a restored image includes:

[0041] Input the image into the degradation function;

[0042] The result processed by the degradation function is added to the random noise to obtain the degraded image;

[0043] An image restoration process is performed based on the degraded image and the precise motion blur kernel to obtain a restored image.

[0044] Optionally, the stitching of the plurality of image blocks includes:

[0045] A weighted average is performed on overlapping blocks among the plurality of image blocks.

[0046] Optionally, the spliced ​​image is input into a pre-trained reconstruction model to obtain a reconstructed image. When the pre-trained reconstruction model is trained,

[0047] The residual between the high-resolution image and the low-resolution image is learned, and the residual image is modeled.

[0048] Optionally, the stitched image is input into a pre-trained reconstruction model to obtain a reconstructed image, and the training of the pre-trained reconstruction model includes:

[0049] Input layer, used to interpolate the image to the target size;

[0050] The middle layer is used to perform convolution calculations on the brightness channel;

[0051] And the output layer is used for image fusion output.

[0052] Optionally, the intermediate layer includes multiple layers of convolution, and each layer of convolution uses a convolution layer with padded pixels.

[0053] In a second aspect, an embodiment of the present invention further discloses an image reconstruction device, comprising:

[0054] A preprocessing module, used for performing block processing on the acquired magnetic resonance image;

[0055] A restoration module is used to process the image after block processing using a blind restoration algorithm to obtain multiple restored image blocks;

[0056] A splicing module, configured to splice a plurality of the image blocks;

[0057] The reconstruction module is used to input the spliced ​​image into the pre-trained reconstruction model to obtain a reconstructed image.

[0058] In a third aspect, an embodiment of the present invention further discloses an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of an image reconstruction method as described in any one of the first aspects.

[0059] The beneficial effects of the embodiments of the present invention are:

[0060] The technical solution disclosed in the embodiment of the present invention performs block prediction and blind restoration on the image, and finally performs splicing and reconstruction on the input pre-trained reconstruction model. The blind restoration can use the estimation of the blur kernel to perform motion blur repair and high-resolution prediction, and perform random trajectory prediction on the motion artifacts of medical images with spatial shift properties, so as to prevent the artifact enhancement of the motion blur itself during the resolution reconstruction process, thereby achieving the purpose of reducing the enhancement effect of motion artifacts.

[0061] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0063] Figure 1 A flow chart of an image reconstruction method provided by an embodiment of the present invention is shown;

[0064] Figure 2 shows a flow chart of a blind restoration algorithm provided by an embodiment of the present invention;

[0065] Figure 3 A schematic diagram of a VDSR provided by an embodiment of the present invention is shown;

[0066] Figure 4a A schematic diagram of an image before segmentation provided by an embodiment of the present invention is shown;

[0067] Figure 4b A schematic diagram of a segmented image provided by an embodiment of the present invention is shown;

[0068] Figure 5 A schematic diagram of a blur kernel corresponding to a predicted trajectory curve provided by an embodiment of the present invention is shown;

[0069] Figure 6 shows a restoration-error plane schematic diagram provided by an embodiment of the present invention;

[0070] Figure 7 The figure shows a restoration and degradation flow chart provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0072] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0073] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediary; or internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0074] Super-resolution;

[0075] Blind deconvolution;

[0076] deep learning;

[0077] PSF (Point Spread Function) is a point spread function. The blur kernel in this article is synonymous with PSF.

[0078] Blind restoration, also known as blind deconvolution, is commonly used in the field of machine vision. Image restoration involves extracting useful information from low-resolution images to identify and remove blur and noise caused by motion artifacts.

[0079] Image restoration is divided into two categories: one is the case where the point spread function is known, also known as non-blind deconvolution image restoration, which is suitable for motion blur of arbitrary trajectory of additive noise and motion blur of simple straight line trajectory. The calculation of this type of image restoration does not involve PSF estimation and has clearer image prior knowledge; the other is the case where the point spread function is unknown, also known as blind deconvolution image restoration. Since there is no prior PSF, it is necessary to use relatively one-sided prior knowledge, use regularization terms to constrain the model, suppress noise, prevent the Gibbs effect of the image from affecting the results too much, and find and predict the PSF.

[0080] Example 1:

[0081] like Figure 1 As shown, an image reconstruction method includes:

[0082] Performing block processing on the acquired magnetic resonance images;

[0083] After the image is acquired, image preprocessing is performed first. For each magnetic resonance image, and considering the spatial shift of the image itself, the original image needs to be divided into blocks for estimating the block convolution kernel.

[0084] The PSF of the magnetic resonance system itself is spatially variable. Therefore, the estimated motion and blur kernel caused by the hardware in this embodiment should not be global. Based on this assumption, the target image needs to be processed in blocks, and the blur kernel needs to be estimated for each block.

[0085] At the same time, since motion artifacts are localized, this embodiment chooses to perform local aliasing processing on the blocks to prevent the unrecognized motion artifacts existing at the edges and the generation of ghost images caused by the blocks.

[0086] The image after block processing is processed using a blind restoration algorithm to obtain multiple restored image blocks;

[0087] splicing a plurality of the image blocks;

[0088] The spliced ​​image is input into the pre-trained reconstruction model to obtain the reconstructed image.

[0089] Optionally, the block-processed image is processed using a blind restoration algorithm to obtain a plurality of restored image blocks, including:

[0090] Perform image modeling and select corresponding parameters based on the collected signals, wherein the parameters are parameters of the model obtained by image modeling;

[0091] Predict motion blur kernel based on image modeling;

[0092] performing an error analysis on the motion blur kernel and creating a restoration-error pair model based on the error analysis;

[0093] Obtaining an accurate motion blur kernel based on the restoration-error pair model;

[0094] Non-blind image restoration is performed based on the precise motion blur kernel to obtain a restored image.

[0095] Optionally, the image modeling and corresponding parameter selection based on the collected signal, wherein the parameters are parameters of the model obtained by image modeling, include:

[0096] After signal acquisition at time t, the image acquired by modeling is:

[0097] z t =k(u t (x)+∈(x)),x∈X,

[0098] refers to the discrete grid of sampling, k is used to normalize the signal to a limited dynamic range u t (x) represents the time-varying and shift-varying Poisson distribution, and ∈(x) represents the time-invariant and shift-invariant Gaussian distribution.

[0099] Optionally, obtaining an accurate motion blur kernel based on the restoration-error pair model includes:

[0100] Choose a time from a limited set of times;

[0101] Obtaining a restoration-error plane based on the time and restoration-error pair model;

[0102] The predicted acquisition signal time is selected based on the restoration-error plane, and the optimal acquisition time solution is obtained through iteration;

[0103] An accurate motion blur kernel is obtained based on the optimal acquisition time solution.

[0104] Optionally, the restoration-error pair model is:

[0105]

[0106] y is the input image, λ and σ are the user input parameters of the two types of kernel distributions, r is the expected restoration error of the restoration-error pair model, θ1, θ2, ..., θ n is the descriptor of the motion blur kernel, is a statistical method, t is the signal acquisition time, h t is the motion blur kernel, H t is the set of models used to calculate the restoration error.

[0107] Optionally, performing non-blind image restoration based on the precise motion blur kernel to obtain a restored image includes:

[0108] Input the image into the degradation function;

[0109] The result processed by the degradation function is added to the random noise to obtain the degraded image;

[0110] An image restoration process is performed based on the degraded image and the precise motion blur kernel to obtain a restored image.

[0111] Optionally, the stitching of the plurality of image blocks includes:

[0112] A weighted average is performed on overlapping blocks among the plurality of image blocks.

[0113] Optionally, the spliced ​​image is input into a pre-trained reconstruction model to obtain a reconstructed image. During training, the pre-trained reconstruction model learns the residual between the high-resolution image and the low-resolution image and models the residual image.

[0114] Optionally, the stitched image is input into a pre-trained reconstruction model to obtain a reconstructed image, and the training of the pre-trained reconstruction model includes:

[0115] Input layer, used to interpolate the image to the target size;

[0116] The middle layer is used to perform convolution calculations on the brightness channel;

[0117] And the output layer is used for image fusion output.

[0118] Optionally, the intermediate layer includes multiple layers of convolution, and each layer of convolution uses a convolution layer with padded pixels.

[0119] Image preprocessing is to process the acquired magnetic resonance images into blocks;

[0120] After the image is acquired, image preprocessing is performed first. For each magnetic resonance image, and considering the spatial shift of the image itself, the original image needs to be divided into blocks for estimating the block convolution kernel.

[0121] The PSF of the magnetic resonance system itself is spatially variable. Therefore, the estimated motion and blur kernel caused by the hardware in this embodiment should not be global. Based on this assumption, the target image needs to be processed in blocks, and the blur kernel needs to be estimated for each block.

[0122] At the same time, since motion artifacts are localized, this embodiment chooses to perform local aliasing processing on the blocks to prevent the unrecognized motion artifacts existing at the edges and the generation of ghost images caused by the blocks.

[0123] Blind restoration algorithm such as Figure 2 As shown,

[0124] Since there is no prior PSF as prior knowledge, the blind deconvolution method is used in this embodiment to perform blind restoration of the low-resolution image.

[0125] Blind restoration estimates the PSF by sampling the trajectory of a continuous trajectory curve on a regular pixel grid on a discrete grid using sub-pixel linear interpolation using a Markov random process. This embodiment considers both signal-dependent and signal-independent PSF distributions in the image modeling, and also accounts for the impact of signal acquisition time on the trajectory and signal blur.

[0126] Step 1. Image modeling and parameter selection. The images collected by modeling represent the time-varying and shift-varying Poisson distribution and the time-invariant and shift-invariant Gaussian distribution respectively.

[0127] Step 2. Motion blur kernel prediction. This embodiment uses the Markov Chain Monte Carlo (MCMC) method to derive the error model constraint to guide the point spread function restoration. That is, at this time h t It can be calculated from H t The average expectation of a large number of random motion PSFs estimated in . By sampling continuous trajectories on a (regular) pixel grid, we obtain a set of Each trajectory consists of the position of a particle moving randomly in two dimensions over a continuous region.

[0128] Step 3. Error analysis and restoration - Error pair model establishment, error analysis of motion blur kernel, due to motion blur PSF h t It cannot be accurately predicted, so it can be regarded as a random process, and the error analysis should be carried out in a statistical way. Combined with the error analysis, a restoration-error pair model is created. The restoration-error pair model includes h t ~H t All expected restoration errors r,θ1,θ2,…,θ n is a descriptor of the PSF. The predicted acquisition time is selected based on the restoration-error plane, and the optimal acquisition time solution is finally obtained through iteration. The blur kernel PSF can be predicted based on this optimal acquisition time solution.

[0129] Step 4. Non-blind image restoration: The problem is transformed into a simpler image deconvolution problem. Non-blind deconvolution models can be implemented using RL methods, etc.

[0130] Image block stitching:

[0131] After preprocessing, the image blocks are sequentially fused, and weighted averaging is required for overlapping blocks to prevent artifact enhancement caused by motion artifacts at the edge of the blocks.

[0132] Super-resolution reconstruction based on deep learning:

[0133] The core of this step is to input a low-resolution image into the recursive residual network and output a high-resolution image.

[0134] Since the low-resolution and high-resolution images themselves have not changed, the two are very similar, and the low-frequency information carried by the low-resolution image is similar to that of the high-resolution image. So when training, VDSR is selected. In order to save time, the network only learns the residual between high-resolution and low-resolution, and models the residual images of the two. This makes the receptive field of the network of this structure larger, so that it can utilize contextual information of a larger area. At the same time, by only modeling the residual between LR and HR, there is no need to spend more time on the low-frequency part, which makes the convergence faster and the learning rate higher. At the same time, residual learning and gradient clipping can solve the problem of gradient explosion caused by the large number of network layers. The entire network structure is as follows Figure 3 shown.

[0135] The network is mainly divided into three parts:

[0136] 1. Input layer: first interpolate the image to the target size and input it into the network.

[0137] 2. In the middle layer, 20 layers of 3*3 convolution are performed for the brightness channel.

[0138] 3. Output layer, image fusion output

[0139] During the model training process, small-batch gradient descent and adaptive gradient clipping are used, and the loss function is the mean square error of the residual between the low-resolution image and the high-resolution predicted image.

[0140] This embodiment uses blind restoration as the motion blur preprocessing process for magnetic resonance super-resolution reconstruction for the first time. During training, for the sake of training effect, a low-resolution input is used that is first degraded and then restored using a high-resolution image, and the high-resolution original image is used as ground truth (annotated data). In specific applications, it is necessary to consider the existence of random factors such as motion artifacts, which will make the network itself less generalizable. This embodiment can perform random trajectory prediction on motion artifacts in medical images with spatially varying properties, preventing the artifact enhancement of motion blur itself during the resolution reconstruction process.

[0141] At the same time, by using a randomly predicted motion blur kernel for blind restoration, the acquisition of the blur kernel, resulting from the interaction of patient motion blur, hardware-induced noise and artifacts, and rapid acquisition time, is considered a random process. This addresses the existing problem of difficulty in predicting the PSF of medical images when blind restoration algorithms are applied to magnetic resonance imaging. This embodiment considers the acquisition of the PSF, resulting from the interaction of patient motion blur, hardware-induced noise and artifacts, and rapid acquisition time, as a random process, and uses the algorithm described in this embodiment to predict the motion blur kernel of the PSF, thereby accurately predicting the PSF.

[0142] During the preprocessing process, the image is novelly divided into blocks. Considering the randomness and locality of motion artifacts in medical images, the original image is divided into blocks and the random blur kernel is predicted separately to more accurately locate local motion blur and artifacts.

[0143] This embodiment takes into account exposure time, interaction with image noise, and randomness of the signal. Each specific restoration-error model allows identification of the correct acquisition strategy to maximize the performance of the corresponding deblurring algorithm.

[0144] In this embodiment, the model regards the point spread function trajectory as a random process, adopts the Monte Carlo method, and expresses the restoration performance as the restoration error expectation based on the motion randomness descriptor and the exposure time.

[0145] If the blur of the image is predicted and blindly restored in the preprocessing stage, and finally spliced ​​together and used as a training dataset, blind restoration is a type of technology that can use blur kernel estimation to perform motion blur repair and high-resolution prediction. Since blind restoration has been proven to remove image blur and edge loss caused by motion to a certain extent, this method can reduce detail loss and prevent the occurrence of problems such as image edge blur based on the original solution.

[0146] This embodiment can be used for post-processing of various magnetic resonance images, and can solve the problems of existing super-resolution image reconstruction methods for magnetic resonance images, such as long iteration time, difficulty in eliminating motion artifacts, blurred image edges, and loss of details, without making additional hardware modifications.

[0147] Example 2:

[0148] 1. Image preprocessing

[0149] After the image is acquired, the image is first preprocessed. For each magnetic resonance image, and considering the spatial shift of the image itself, the original image needs to be divided into blocks for the estimation of the block convolution kernel. Figure 4a and Figure 4b shown.

[0150] The PSF of the magnetic resonance system itself is spatially variable. Therefore, the estimated motion and blur kernel caused by the hardware in this embodiment should not be global. Based on this assumption, the target image needs to be processed in blocks, and the blur kernel needs to be estimated for each block.

[0151] According to the characteristics of magnetic resonance, there is a certain time interval between each phase encoding line when sequence encoding is performed, while the sampling time interval of the frequency encoding line can be ignored to a certain extent. Due to the abnormal accumulation of phase and the existence of acquisition time span, the appearance of motion artifacts is common in the phase encoding direction. Therefore, according to Figure 4a The image is divided into blocks.

[0152] The reason why global direct processing is not chosen is that although direct global processing can speed up the processing to a certain extent, reduce the preprocessing steps, and reduce the computational complexity, it ignores the effect of the spatial variability of the PSF itself applied to the global image relative to the magnetic resonance image itself.

[0153] At the same time, since the motion artifacts are local, this embodiment chooses to perform local aliasing processing on the blocks, such as Figure 4b This prevents the unrecognition of motion artifacts at the edge and the generation of ghost images due to blocking.

[0154] 2. Blind restoration algorithm

[0155] Since there is no prior PSF as prior knowledge, the blind deconvolution method is used in this embodiment to perform blind restoration of the low-resolution image.

[0156] The reasons why blind restoration is more feasible are:

[0157] (1) As a natural image, the gradient distribution of magnetic resonance images conforms to a heavy-tailed distribution. However, images with motion blur will have a certain difference in gradient distribution relative to the original clear image. Therefore, there is prior knowledge about the prediction of the blur kernel, which can be used as a quantitative reference for the optimization results.

[0158] (2) Due to the sparse effects of medical images, that is, there are more zero points than non-zero points, the blur kernel caused by motion artifacts is also sparse and non-negative.

[0159] With these characteristics as key prior information, the shape of the blur kernel can be estimated, thus making non-blind restoration possible.

[0160] The blind restoration process is described below:

[0161] This embodiment estimates the PSF by using sub-pixel linear interpolation to sample the motion trajectory of a continuous trajectory curve on a regular pixel grid on a discrete grid using a Markov random process. Image modeling considers both signal-dependent and signal-independent PSF distributions, and also considers the impact of signal acquisition time on motion trajectory and signal blur.

[0162] Step 1. Image modeling and parameter selection:

[0163] First, after time t of signal acquisition, the image acquired by modeling is:

[0164] z t =k(u t (x)+∈(x)),x∈X,

[0165] Refers to the discrete grid of sampling. k is used to normalize the signal to a limited dynamic range for subsequent processing. t (x) and ∈(x) represent the time-varying and shift-varying Poisson distribution and the time-invariant and shift-invariant Gaussian distribution, respectively. They can be expressed as and∈(x)~N(0,σ 2 ). The y function represents the restored image without motion blur and artifacts.

[0166] Step 2. Motion blur kernel prediction

[0167] The initial assumption of this embodiment is that for the entire magnetic resonance image in a block region, the shift s(t) caused by motion is constant for the entire image block.

[0168] The Poisson distribution in the above formula is a time-varying and space-varying model, and its parameters can be written as:

[0169]

[0170] Among them h t represents the motion blur kernel, since h t >0, so

[0171] This embodiment uses the Markov Chain Monte Carlo (MCMC) method to derive the error model constraint to guide the point spread function restoration. That is, at this time h t It can be calculated from H t The average expectation of a large number of random motion PSFs estimated in . By sampling continuous trajectories on a (regular) pixel grid, we obtain a set of Each trajectory consists of the position of a particle moving randomly in two dimensions within a continuous region, such as Figure 5 As shown, it represents the motion blur kernel corresponding to the trajectory curve.

[0172] Step 3. Error analysis and restoration - error pair model establishment

[0173] Perform error analysis on the motion blur kernel. Since the motion blur PSF h t It cannot be accurately predicted, so it can be regarded as a random process, and the error analysis should be done in a statistical way. The error function used is RMSE (Root Mean Squared Error) to guide the establishment of the subsequent restoration-error model, which is consistent with the image z t The corresponding RMSE r(z t ,y) is defined as:

[0174]

[0175] M is the grayscale range of the image, #X is the number of pixels, Refers to a perfectly restored image.

[0176] Combined with error analysis, a restoration-error pair model is created:

[0177]

[0178] y is the input image, λ and σ are the user input parameters of the two types of kernel distributions, and the restoration-error pair model includes h t ~H t All expected restoration errors r,θ1,θ2,…,θ n is the descriptor of PSF. It is a class of statistical methods, such as adaptive parametric or nonparametric smoothing filters, used to estimate the conditional expectation from several realizations of restoration-error pairs.

[0179] The smoothing value at a certain time point t is defined as the restoration-error plane R at time t λ,σ,y (…,t), the schematic diagram is as follows Figure 6 shown.

[0180] For any initial value t', the restoration-error plane can be calculated and expressed. To define the restoration error model, we only need to consider a selected finite set of acquisition signal times t1, t2, ..., t n , and interpolate the error surfaces for different t values.

[0181] When different signal acquisition times are selected, in actual practice, the PSF obtained will vary with different time nodes. In this case, replacing time t, the restoration-error plane becomes a single variable function about time t:

[0182] R λ,σ,y (t) = R λ,σ,y (θ1(t),θ2(t),…,θ n (t),t),

[0183] At this point, the predicted acquisition signal time can be selected based on the restoration-error plane, and the optimal acquisition time solution is finally obtained by iteration: T′=argmin[R λ,σ,y (t)],

[0184] The PSF can be predicted based on the optimal acquisition time solution.

[0185] Step 4. Non-blind image restoration:

[0186] After estimating the PSF, the image restoration problem is transformed into a simpler non-blind image restoration method.

[0187] The image degradation process can be regarded as a system consisting of a degradation function and additive noise acting on the input image, and the two work together to produce a degraded image.

[0188] When the degradation model is described as a linear system, the degradation process can be expressed as:

[0189]

[0190] N is the random noise caused by various factors during the sampling process.

[0191] Therefore, the restoration process, as the inverse process of image degradation, can be combined with the degradation process and expressed as Figure 7 shown.

[0192] The most common RL algorithm is selected. The RL (Richardson-Lucy) algorithm is an iterative nonlinear restoration algorithm that uses an iterative expectation maximization process and a Poisson distribution model to model the problem from z t The deconvolution image is obtained from . The ideal RMSE can be calculated by selecting the number of iterations.

[0193] The image non-blind deconvolution model is as follows:

[0194]

[0195] 3. Image block stitching

[0196] In the sequential fusion of image blocks, weighted averaging is required for the overlapping blocks to prevent the artifact enhancement problem caused by the presence of motion artifacts at the edge of the blocks.

[0197] 4. Super-resolution reconstruction based on deep learning:

[0198] The core of this step is to input a low-resolution image into the recursive residual network and output a high-resolution image.

[0199] Since the low-resolution and high-resolution images themselves remain unchanged and are very similar, the low-frequency information carried by the low-resolution image is similar to that of the high-resolution image. Therefore, VDSR is selected for training. To save time, this network only learns the residual between the high-resolution and low-resolution images and models the residual images of the two. This increases the receptive field of the network structure, allowing it to utilize contextual information from a larger area. Furthermore, by modeling only the residual between the LR and HR images, more time is not spent on the low-frequency portion, resulting in faster convergence and a higher learning rate. Furthermore, residual learning and gradient clipping can address the gradient explosion problem caused by the large number of network layers.

[0200] The entire network structure is as follows Figure 3 shown.

[0201] The network is mainly divided into three parts:

[0202] 1. Input layer: first interpolate the image to the target size and input it into the network.

[0203] The first layer operates on the input image, performs bicubic interpolation, and upsamples the LR (Low Resolution) image so that when the subsequent network is processed, it processes the feature map of the target size. The processed single-channel image is output as a 64-channel feature map through 64 different convolution kernels with a size of 3*3 and input into the network.

[0204] 2. In the middle layer, 20 layers of 3*3 convolution are performed for the brightness channel.

[0205] Each convolution layer uses a 3x3 convolution layer with padding (padded pixels). This network uses small convolution kernels, resulting in more network layers while reducing the number of parameters. Traditional convolution operations would result in poor results due to the small receptive field of boundary pixels. This network performs zero padding on the image after each convolution, preserving the original feature map size and restoring the original size, thus addressing the issue of network depth. A network with a depth of d can have a receptive field of (2d+1)*(2d+1). As the network depth increases, subsequent layers have larger receptive fields.

[0206] A ReLU is added after each convolutional layer to enhance the nonlinearity of the model and prevent gradient explosion. From the second layer to the last layer, the filter size of the previous layer is 3*3*64.

[0207] 3. Output layer, image fusion output

[0208] The final layer is used for image reconstruction, fusing the 64 channels and converting them back into a single-channel image. After obtaining the image details predicted by the network, it is added to the low-resolution input network that has been interpolated and upscaled to obtain the output.

[0209] During the model training process, small-batch gradient descent and adaptive gradient clipping are used. To avoid gradient explosion, the gradient is limited to between -G / r and G / r, where G and r are the gradient and learning rate respectively.

[0210] The loss function of the VSDN model is the mean square error of the residual between the low-resolution image and the high-resolution predicted image. The loss function can be expressed as follows:

[0211]

[0212] in, is the low-resolution image after bicubic interpolation, is a high-resolution image, is the residual high-resolution image prediction

[0213] In the blind convolution process of this embodiment, the non-blind deconvolution method can be replaced by other methods such as Weiner deconvolution, anisotropic LPA-ICI deconvolution, and sparse image prior deconvolution.

[0214] The super-resolution method based on deep learning can be replaced by other networks such as SRCNN, FSRCNN, and DRCN.

[0215] Example 3:

[0216] An image reconstruction device, comprising:

[0217] A preprocessing module, used for performing block processing on the acquired magnetic resonance image;

[0218] A restoration module is used to process the image after block processing using a blind restoration algorithm to obtain multiple restored image blocks;

[0219] A splicing module, configured to splice a plurality of the image blocks;

[0220] The reconstruction module is used to input the spliced ​​image into the pre-trained reconstruction model to obtain a reconstructed image.

[0221] Example 4:

[0222] In addition, an embodiment of the present invention also provides an electronic device, including a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and runnable on the processor. The transceiver, the memory, and the processor are respectively connected via a bus. When the computer program is executed by the processor, the various processes of the above-mentioned image reconstruction method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0223] Embodiment 5:

[0224] In addition, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned image reconstruction method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0225] Computer-readable storage media include: permanent and non-permanent, removable and non-removable media, which are tangible devices that can retain and store instructions for use by instruction execution devices. Computer-readable storage media include: electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, and any suitable combination of the above. Computer-readable storage media include: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette storage, magnetic disk storage or other magnetic storage devices, memory sticks, mechanical encoding devices (such as punched cards or raised structures with grooves in which instructions are recorded), or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined in the embodiments of the present invention, computer-readable storage media does not include temporary signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (such as light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0226] In the several embodiments provided in this application, it should be understood that the disclosed devices, electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be an electrical, mechanical or other form of connection.

[0227] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, and may be located in a single location or distributed across multiple network units. Some or all of these units may be selected based on actual needs to address the issues addressed by the embodiments of the present invention.

[0228] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0229] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (including: a personal computer, a server, a data center or other network device) to perform all or part of the steps of the method described in each embodiment of the present invention. The above-mentioned storage medium includes the various media that can store program codes as listed above.

[0230] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technical solution that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. An image reconstruction method, characterized in that: include: Performing block processing on the acquired magnetic resonance images; The image after block processing is processed using a blind restoration algorithm to obtain multiple restored image blocks; splicing a plurality of the image blocks; Input the spliced ​​image into the pre-trained reconstruction model to obtain a reconstructed image; The image after block processing is processed using a blind restoration algorithm to obtain multiple restored image blocks, including: Perform image modeling and select corresponding parameters based on the collected signals, wherein the parameters are parameters of the model obtained by image modeling; Predict motion blur kernel based on image modeling; performing an error analysis on the motion blur kernel and creating a restoration-error pair model based on the error analysis; Obtaining an accurate motion blur kernel based on the restoration-error pair model; Performing non-blind restoration of the image based on the precise motion blur kernel to obtain a restored image; The image modeling and corresponding parameter selection based on the collected signal, wherein the parameters are parameters of the model obtained by image modeling, include: After signal acquisition at time t, the image acquired by modeling is: ， is the discrete grid of sampling, Used to normalize the signal to a limited dynamic range, represents the time-varying and shift-varying Poisson distribution, represents a time-invariant and shift-invariant Gaussian distribution.

2. The image reconstruction method according to claim 1, wherein: The method of obtaining an accurate motion blur kernel based on the restoration-error pair model includes: Choose a time from a limited set of times; Obtaining a restoration-error plane based on the time and restoration-error pair model; The predicted acquisition signal time is selected based on the restoration-error plane, and the optimal acquisition time solution is obtained through iteration; An accurate motion blur kernel is obtained based on the optimal acquisition time solution.

3. The image reconstruction method according to claim 1, wherein: The restoration-error pair model is: ， is the input image, are the user input parameters for the two types of kernel distributions, r is the expected restoration error of the restoration-error pair model, is the descriptor of the motion blur kernel, is a statistical method, t is the signal acquisition time, is the motion blur kernel, is the set of models used to calculate the restoration error.

4. The image reconstruction method according to claim 1, wherein: The non-blind restoration of the image based on the precise motion blur kernel to obtain the restored image includes: Input the image into the degradation function; The result processed by the degradation function is added to the random noise to obtain the degraded image; An image restoration process is performed based on the degraded image and the precise motion blur kernel to obtain a restored image.

5. The image reconstruction method according to claim 1, wherein: The step of splicing the plurality of image blocks comprises: A weighted average is performed on overlapping blocks among the plurality of image blocks.

6. The image reconstruction method according to claim 1, wherein: The spliced ​​image is input into a pre-trained reconstruction model to obtain a reconstructed image. When the pre-trained reconstruction model is trained, The residual between the high-resolution image and the low-resolution image is learned, and the residual image is modeled.

7. The image reconstruction method according to claim 1, wherein: The spliced ​​image is input into a pre-trained reconstruction model to obtain a reconstructed image, and the pre-trained reconstruction model training includes: Input layer, used to interpolate the image to the target size; The middle layer is used to perform convolution calculations on the brightness channel; And the output layer is used for image fusion output.

8. The image reconstruction method according to claim 7, characterized in that: The intermediate layer includes multiple layers of convolution, and each layer of convolution uses a convolution layer with padded pixels.

9. An image reconstruction device, characterized in that: include: A preprocessing module, used for performing block processing on the acquired magnetic resonance image; A restoration module is used to process the image after block processing using a blind restoration algorithm to obtain multiple restored image blocks; A splicing module, configured to splice a plurality of the image blocks; The reconstruction module is used to input the spliced ​​image into the pre-trained reconstruction model to obtain a reconstructed image; The image after block processing is processed using a blind restoration algorithm to obtain multiple restored image blocks, including: Perform image modeling and select corresponding parameters based on the collected signals, wherein the parameters are parameters of the model obtained by image modeling; Predict motion blur kernel based on image modeling; performing an error analysis on the motion blur kernel and creating a restoration-error pair model based on the error analysis; Obtaining an accurate motion blur kernel based on the restoration-error pair model; Performing non-blind restoration of the image based on the precise motion blur kernel to obtain a restored image; The image modeling and corresponding parameter selection based on the collected signal, wherein the parameters are parameters of the model obtained by image modeling, include: After signal acquisition at time t, the image acquired by modeling is: ， is the discrete grid of sampling, Used to normalize the signal to a limited dynamic range, represents the time-varying and shift-varying Poisson distribution, represents a time-invariant and shift-invariant Gaussian distribution.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by the processor, the steps of the image reconstruction method according to any one of claims 1 to 8 are implemented.

Images