Self-supervised reconstruction method and device of magnetic resonance image, medium, program product and terminal

Abstract

The invention provides a self-supervised reconstruction method and device for a magnetic resonance image, a medium, a program product and a terminal. The method comprises the following steps: inputting original under-sampling data into a first sampling reconstruction model to generate a first full-sampling image; performing simulation undersampling operation on the first full sampling image to generate simulation undersampling data; inputting the simulation under-sampling data into a second sampling reconstruction model to generate a high-quality second full-sampling image; and constructing a loss function based on the original sampling data and the second full sampling image, and updating and optimizing model parameters. The problem of inconsistent data distribution in the training stage and the reasoning stage in the prior art is solved, meanwhile, the data utilization efficiency and the model generalization ability in the training process are improved, efficient and accurate magnetic resonance image reconstruction can be achieved without depending on a complete image or two collection processes, and the method has remarkable effects in the aspects of reducing artifacts and improving image quality.

Description

Technical Field

[0001] The present application relates to the field of digital image processing, and particularly to a self-supervised reconstruction method, device, medium, program product, and terminal for magnetic resonance images. Background Art

[0002] Magnetic Resonance Imaging (MRI) is an important medical diagnostic technique. However, its data acquisition process is relatively slow. Especially when sampling point by point in the k-space (Fourier space), it takes a long time, which affects the patient's comfort. To accelerate the acquisition, undersampling is usually adopted, that is, reducing the number of sampling points in the k-space. However, undersampling does not satisfy the Nyquist sampling theorem, which will cause aliasing artifacts in the image and affect the diagnostic accuracy. The goal of magnetic resonance reconstruction is to recover a complete artifact-free image from the undersampled aliased image. The prior art faces the following core problems in achieving this goal:

[0003] Traditional supervised learning methods usually rely on paired undersampled images and complete images as training data. However, in many practical scenarios, it is difficult or costly to obtain complete k-space data. For example, in dynamic imaging (such as cardiac imaging), due to the acquisition time limit, it is almost impossible to obtain complete data. In addition, the acquisition of complete images usually requires a longer scanning time, which may lead to an increase in patient motion artifacts and further reduce the image quality. Therefore, traditional methods that rely on complete images as training labels are limited in practical applications.

[0004] At the same time, although self-supervised learning methods (such as SSDU) avoid relying on complete images, by dividing the undersampled data collected once into two non-overlapping subsets, one as the training input and the other as the pseudo-label for training, this data division strategy also has significant differences from the undersampled data input in the actual inference stage. Specifically, the undersampled data in the inference stage directly comes from real acquisition, while the data in the training stage is obtained by artificially dividing the real acquisition data twice, which is not consistent in data characteristics and distribution. In addition, the way of data division may reduce the utilization efficiency of information, further limiting the model performance. These differences and deficiencies together lead to the difficulty for the model to fully adapt to the distribution of real undersampled data in the inference stage, affecting the reconstruction quality and generalization ability. Summary of the Invention

[0005] In view of the above-mentioned disadvantages of the prior art, the purpose of the present application is to provide a self-supervised reconstruction method, device, medium, program product and terminal for magnetic resonance images, which are used to solve the problems of relying on complete images as supervision signals, requiring two acquisitions of the same sample data as training pairs, inconsistent training processes and inference scenarios (such as data distribution differences caused by secondary data partitioning), and low data utilization efficiency, resulting in insufficient network generalization ability.

[0006] To achieve the above object and other related objects, the first aspect of the present application provides a self-supervised reconstruction method for magnetic resonance images, including: obtaining a plurality of original sampling data; sequentially inputting each original sampling data into the first sampling reconstruction model to generate a first fully sampled image; performing a simulated undersampling operation on the first fully sampled image to generate simulated undersampled data; inputting the simulated undersampled data into a second sampling reconstruction model to generate a second fully sampled image; calculating a loss function of the current original sampling data and the second fully sampled image, and performing a model parameter update operation based on the loss function.

[0007] In some embodiments of the first aspect of the present application, the first sampling reconstruction model and the second sampling reconstruction model share parameters.

[0008] In some embodiments of the first aspect of the present application, before inputting the simulated undersampled data into the second sampling reconstruction model, the following steps are further performed: replacing the original sampling data with the corresponding position of the simulated undersampled data according to a preset rule.

[0009] In some embodiments of the first aspect of the present application, the process of performing a simulated undersampling operation on the first fully sampled image to generate simulated undersampled data includes: sequentially performing a Fourier transform operation on the first fully sampled image and multiplying it by a preset sampling mask to generate the simulated undersampled data; wherein, the preset sampling mask is consistent with the acquisition distribution of the original sampling data.

[0010] In some embodiments of the first aspect of the present application, the method includes: performing a gradient clipping operation on the output of the first sampling reconstruction model.

[0011] In some embodiments of the first aspect of the present application, the process of performing a model parameter update operation based on the loss function includes: performing a model parameter update operation on the second sampling reconstruction model based on the loss function.

[0012] To achieve the above and other related objectives, a second aspect of the present application provides a self-supervised reconstruction device for magnetic resonance images, including: a model initialization module: configured to obtain a plurality of original sampling data; a model training module: configured to sequentially input each piece of original sampling data into the first sampling reconstruction model to generate a first fully sampled image; perform a simulated undersampling operation on the first fully sampled image to generate simulated undersampled data; input the simulated undersampled data into the second sampling reconstruction model to generate a second fully sampled image; calculate a loss function between the current original sampling data and the second fully sampled image, and perform a model parameter update operation based on the loss function.

[0013] To achieve the above and other related objectives, a third aspect of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the self-supervised reconstruction method for magnetic resonance images is implemented.

[0014] To achieve the above and other related objectives, a fourth aspect of the present application provides a computer program product, which includes computer program code, and when the computer program code runs on a computer, the computer is enabled to implement the self-supervised reconstruction method for magnetic resonance images.

[0015] To achieve the above and other related objectives, a fifth aspect of the present application provides an electronic terminal, including a memory, a processor, and a computer program stored on the memory; the processor executes the computer program to implement the self-supervised reconstruction method for magnetic resonance images.

[0016] As described above, the self-supervised reconstruction method, device, medium, program product, and terminal for magnetic resonance images of the present application have the following beneficial effects: The present application proposes a two-stage model method for magnetic resonance image reconstruction. By introducing the first sampling reconstruction model and the second sampling reconstruction model, and performing a simulated undersampling operation on the basis of the first fully sampled image, the problem of inconsistent distribution between training data and inference data in the prior art is solved. Specifically:

[0017] First, the present technology application generates a first fully sampled image through a first sampling reconstruction model, utilizing more feature information during the training phase to provide a basis for subsequent processing. This technical feature effectively improves the initial quality of the training data. Secondly, a simulated undersampling operation is adopted to generate a data distribution closer to the inference data for the first fully sampled image, thereby narrowing the difference in data distribution between the training phase and the inference phase, effectively overcoming the problems of inaccurate training data distribution or insufficient information utilization in traditional methods (such as SSDU and supervised learning methods). Furthermore, a second sampling reconstruction model is used to generate a high-quality second fully sampled image from the simulated undersampled data, and a loss function is constructed in combination with the original sampled data for end-to-end optimization of the closed-loop, adjusting the model weights to make the prediction of the training model closer to the real inference scenario. This technical feature enhances the generalization ability and reconstruction accuracy of the model in the inference phase. Finally, the present application does not need to rely on real complete images or two data acquisitions, directly generates a dataset that conforms to the inference scenario from single-time undersampled data, and through the collaboration of the entire phased model and efficient data utilization, significantly solves the problems of high dependence on complete data and low information utilization efficiency in traditional supervised and self-supervised methods. Therefore, through the collaboration of the phased model, simulated undersampling operation, and closed-loop training optimization mechanism, the present application effectively solves the problem of the difference in data distribution between training data and inference data, improves the accuracy and robustness of magnetic resonance image reconstruction, and significantly improves the generalization ability in high-complexity scenarios compared with the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 FIG. shows a schematic flow chart of an embodiment of the self-supervised reconstruction method for magnetic resonance images of the present application.

[0019] Figure 2 FIG. shows a schematic flow chart of another embodiment of the self-supervised reconstruction method for magnetic resonance images of the present application.

[0020] Figure 3 FIG. shows a schematic diagram of the distribution of training data and inference data in the prior art of the present application.

[0021] Figure 4 FIG. shows a schematic diagram of the distribution of simulation data and inference data in an embodiment of the self-supervised reconstruction method for magnetic resonance images of the present application.

[0022] Figure 5 FIG. shows a schematic structural diagram of an embodiment of the self-supervised reconstruction device for magnetic resonance images of the present application.

[0023] Figure 6 FIG. shows a schematic structural diagram of an embodiment of the self-supervised reconstruction terminal for magnetic resonance images of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] The following specific examples illustrate the implementation manners of the present application. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0025] Before further elaborating on the present application, the nouns and terms involved in the embodiments of the present application are described. The nouns and terms involved in the embodiments of the present application are applicable to the following explanations:

[0026] <1> Undersampling: In magnetic resonance imaging (MRI), a method to accelerate the data acquisition process by reducing the number of sampling points in the k-space (Fourier space). Although undersampling can shorten the acquisition time, it will cause aliasing artifacts in the image because it violates the Nyquist theorem, and a reconstruction algorithm is needed to restore the complete image.

[0027] <2> Mask: A pattern for selective sampling in the k-space, usually represented in the form of a binary matrix, used to indicate which k-space points are sampled and which are ignored. The design of the mask directly affects the effect of undersampling and the quality of the reconstructed image. Common mask types include random masks, Gaussian masks, regular masks, and masks based on prior knowledge, etc.

[0028] <3> k-space: The mathematical space used to store raw data in magnetic resonance imaging. The data in it can be converted into a magnetic resonance image in the image space through Fourier transform. The sampling pattern of the k-space directly affects the resolution and artifact situation of the image, and it is the core concept in MRI data acquisition and reconstruction.

[0029] <4> Aliasing artifact: Under the condition of undersampling, a phenomenon of overlap or distortion appears in the image due to insufficient sampling points. Aliasing artifacts will reduce the readability and diagnostic accuracy of the image, and need to be removed through reconstruction algorithms (such as compressive sensing or deep learning).

[0030] <5> Random simulation mask: A k-space sampling mask created by random generation algorithms such as Gaussian distribution and Poisson distribution, used to simulate the data acquisition mode under the condition of undersampling. Random simulation masks are often used to train reconstruction models to simulate the distribution characteristics of real undersampled data, thereby improving the generalization ability of the models.

[0031] <6>The Nyquist sampling theorem states that in order to accurately reconstruct a signal, the sampling frequency must be at least twice the highest frequency of the signal. In MRI, if the number of sampling points is below the requirements of the Nyquist sampling theorem, aliasing artifacts will occur, so a reconstruction algorithm is needed to restore the complete image.

[0032] <7>The Fourier transform: A mathematical tool that converts a signal from the time domain (or spatial domain) to the frequency domain, and is used in MRI to convert image spatial data into k-space data. The Fourier transform is the basis for MRI image reconstruction, and its inverse transform is used to generate an image from k-space data.

[0033] For ease of understanding the embodiments of the present application, first in combination with Figure 1 and Figure 2 will be described in detail. Figure 1 FIG. shows a schematic flow chart of a self-supervised reconstruction method for magnetic resonance images in an embodiment of the present application. The self-supervised reconstruction method for magnetic resonance images in this embodiment mainly includes the following steps:

[0034] Step S11: Obtain a plurality of original sampling data.

[0035] The present application mainly targets scenarios where deep learning network training does not rely on fully sampled data. In this embodiment, the original sampling data is non-fully sampled data actually collected. Since magnetic resonance images need to be collected point by point in k-space (Fourier space), full sampling takes a long time, which not only affects patient comfort but also may degrade image quality due to patient movement. Therefore, the acquisition is usually accelerated by reducing the number of sampling lines, thereby obtaining non-fully sampled data, which is the original sampling data in the present application.

[0036] In an embodiment of the present application, the input of the first sampling reconstruction model is the original sampling data, and the output is a simulated fully sampled image; the input of the second sampling reconstruction model is the simulated fully sampled image after a simulated undersampling operation, and the output is a fully sampled image after re-reconstruction. Subsequently, an initialization operation is performed on the first sampling reconstruction model and the second sampling reconstruction model.

[0037] It should be noted that the present application does not strictly limit the specific architecture of the model, and a suitable neural network model can be selected according to actual needs, such as a convolutional neural network (CNN), a Transformer, etc. CNN is good at extracting local features and is suitable for image processing tasks; Transformer is based on the self-attention mechanism and is suitable for dealing with long-range dependencies in complex tasks.

[0038] In this embodiment, after obtaining a plurality of original sampling data, an initialization operation is performed on the second sampling reconstruction model. The initialization operation includes, but is not limited to, methods such as random initialization and pre-trained weight loading to accelerate convergence and improve performance.

[0039] As Figure 2 shown, the core of this application lies in the method of using a cascaded network (Unrolled). By combining the first sampling reconstruction model and the second sampling reconstruction model, alternating optimization is performed between data consistency and the regularization term, thereby achieving high-quality reconstruction of magnetic resonance images. Data consistency optimization ensures that the reconstruction result is physically consistent with the original measurement data, while regularization term optimization effectively suppresses noise and artifacts by introducing sparsity or smoothness constraints. This method of alternating optimization not only ensures data accuracy but also significantly improves the clarity and details of the reconstructed image. At the same time, it has high flexibility and can adjust the optimization steps and parameters according to task requirements.

[0040] Step S12: Sequentially input each original sampling data into the first sampling reconstruction model to generate a first fully sampled image.

[0041] In an embodiment of this application, the first sampling reconstruction model can be a pre-trained network model that can directly map from an undersampled image to a fully sampled image or gradually optimize (the first sampling reconstruction model and the second sampling reconstruction model share parameters; the first reconstruction model can also be constructed by means of a momentum encoder in contrastive learning) the mapping from an undersampled image to a fully sampled image. It should be noted that this application does not limit the specific structural type of the first sampling reconstruction model, but proposes an optimization method based on existing training models, aiming to reduce the performance difference between training data and actual application data. The first fully sampled image is only used as simulated fully sampled data in this embodiment. Specifically, the first fully sampled image will undergo an undersampling operation and then be used as the input for the second model. In this way, the distribution difference of training samples can be effectively reduced, ensuring that the trained image reconstruction mapping model has higher stability.

[0042] Step S13: Perform a simulated undersampling operation on the first fully sampled image to generate simulated undersampled data.

[0043] In an embodiment of this application, the process of performing a simulated undersampling operation on the first fully sampled image to generate simulated undersampled data includes: sequentially performing a Fourier transform operation on the first fully sampled image and multiplying it by a preset sampling mask to generate the simulated undersampled data; wherein, the preset sampling mask is consistent with the acquisition distribution of the original sampling data.

[0044] In this embodiment, a Fourier transform operation is performed on the first fully sampled image to transform it from the image space to the k-space (Fourier space). The Fourier transform is the core operation for the transformation from the spatial domain to the frequency domain in magnetic resonance imaging (MRI). As the representation of the frequency domain, the k-space can directly reflect the frequency domain characteristics of the image and provide a basis for subsequent sampling operations. Subsequently, a preset sampling mask is multiplied point by point with the k-space data. The sampling mask is a binary matrix used to simulate the undersampling pattern in the actual acquisition process. A "1" in the mask indicates that the k-space point is sampled, and a "0" indicates that it is not sampled. The design of the preset sampling mask is consistent with the acquisition distribution of the original sampling data to ensure that the simulated undersampling process can truly reflect the actual acquisition conditions.

[0045] Furthermore, the simulated undersampled data generated through the above operations is partial sampled data in the k-space, and its sampling distribution is consistent with that of the real undersampled data. Specifically, the simulated undersampled data transforms the first fully sampled image to the k-space through the Fourier transform and combines with the preset sampling mask for sampling operations to ensure that the distribution of the generated undersampled data in the k-space is exactly the same as the actual acquisition conditions. This consistency is mainly reflected in the sampling pattern matching: the design of the preset sampling mask is based on the sampling distribution of the actual acquisition data, including the sampling trajectory (such as radial, spiral, or Cartesian sampling) and the sampling density (such as high-density sampling in the central region and low-density sampling in the edge region). In this way, the simulated undersampled data can accurately reflect the undersampling characteristics in the actual acquisition and provide high-fidelity input data for the subsequent training of the model.

[0046] In an embodiment of the present application, a gradient clipping operation is performed on the output of the first sampling reconstruction model.

[0047] In this embodiment, the process of performing the gradient clipping operation on the output of the first sampling reconstruction model includes: the parameter updates of the first reconstruction network and the second reconstruction network should not be overly coupled when performing the gradient clipping operation on the output of the first sampling reconstruction model, such as freezing the parameters when using a pre-trained model; performing the gradient clipping operation when sharing parameters, and taking corresponding gradient isolation operations in other ways. The gradient isolation operation is used to block the update of the parameters of the first sampling reconstruction model (ReconNet) during the gradient backpropagation process, so as to ensure the generation of independent and stable simulated undersampled data samples during the training process. Specifically, the gradient isolation operation freezes the gradient calculation of the simulated undersampled data, making it regarded as a fixed input in the subsequent training of the model without affecting the parameters of the first reconstruction network.

[0048] It should be noted that the gradient isolation operation effectively isolates the parameter update paths of the first sampling and reconstruction model and the subsequent models by blocking the reverse propagation of gradients, avoiding mutual interference between the two. This isolation ensures the stability and consistency of the first sampling and reconstruction model when generating simulated undersampled data, while allowing the second sampling and reconstruction model to focus on optimizing its own reconstruction performance. In addition, by blocking the gradient transfer, the gradient cut-off operation makes the generated simulated undersampled data independent during the training process, effectively avoiding excessive coupling between the training data and the model parameters, thereby improving the robustness of the model in practical applications.

[0049] Step S14: Input the simulated undersampled data into the second sampling and reconstruction model to generate a second fully sampled image.

[0050] In an embodiment of the present application, before inputting the simulated undersampled data into the second sampling and reconstruction model, the following steps are further performed: replacing the original sampling data with the corresponding position of the simulated undersampled data according to a preset rule.

[0051] In this embodiment, replacing the original sampling data with the corresponding position of the simulated undersampled data according to a preset rule adds an additional correction process to the simulated undersampled data as a simulation image, that is, replacing the data at the corresponding position in the network output with the actually collected data, thereby reducing the difference between the image used for training (original sampling data) and the simulation image (simulated sampling data), and then solving the problem of the degradation of the reconstruction model performance caused by the inconsistent distribution between the simulation data and the actually collected data. By introducing real information, the quality of the simulated undersampled data and the generalization ability of the reconstruction model are significantly improved.

[0052] Among them, the preset rules include but are not limited to: low-frequency region extraction rule, local feature extraction rule, and high signal-to-noise ratio region extraction rule. The low-frequency region extraction rule preferentially extracts the data in the k-space central region (low-frequency component) and replaces the extracted partial image data with the corresponding position of the simulated undersampled data. Its advantage is that it can retain the main structural information of the image and significantly improve the overall quality and stability of the reconstructed image. The local feature extraction rule preferentially extracts the region data containing important anatomical structures or details according to the spatial domain features of the image and replaces the extracted partial image data with the corresponding position of the simulated undersampled data. Its advantage is that it can retain the key local information of the image and improve the detail resolution and diagnostic value of the reconstructed image. The high signal-to-noise ratio region extraction rule preferentially extracts the region data with a high signal-to-noise ratio in the original sampling data and replaces the extracted partial image data with the corresponding position of the simulated undersampled data. Its advantage is that it can effectively reduce the influence of noise on the reconstructed image and improve the clarity and diagnostic accuracy of the image.

[0053] Step S15: Calculate the loss function of the current original sampling data and the second full-sampling image, and perform a model parameter update operation based on the loss function.

[0054] In an embodiment of the present application, the first sampling reconstruction model and the second sampling reconstruction model share parameters, where the first sampling reconstruction model can provide a relatively stable (pre-trained model) or gradually optimized (the first sampling reconstruction model and the second sampling reconstruction model share parameters; the first reconstruction model can also be constructed by means of a momentum encoder in contrastive learning) mapping from the undersampled image to the fully sampled image.

[0055] In this embodiment, the pre-trained model refers to a model in which the trained method can provide a relatively stable mapping from the undersampled image to the fully sampled image. The training of the model can be initially trained by means of existing self-supervised methods or traditional compressive sensing algorithms, etc.

[0056] In this embodiment, parameter sharing means that the first sampling reconstruction model and the second sampling reconstruction model use the same set of parameters during training and inference. Specifically, the two models share the same set of weight matrices and bias terms, but are optimized under different tasks or data inputs. Specifically, the first sampling reconstruction model is used to process the original undersampled data to generate a preliminary reconstruction result; the second sampling reconstruction model then performs refined correction based on the preliminary result and the original data to generate a mapping from the undersampled MRI image data to the fully sampled MRI image data. In addition to parameter sharing, the first sampling reconstruction model can also be regarded as a momentum encoder, and its parameters are updated by the method of adding the full average of the parameters of the second sampling reconstruction model and itself. The initialization of the parameters of the first sampling reconstruction model can be zero initialization or random initialization.

[0057] Furthermore, on the basis of parameter sharing, this embodiment also introduces a Stop Gradient mechanism to control the gradient propagation of the entire image reconstruction network and avoid mutual interference between the two models during joint training. Specifically, during the training of the first sampling reconstruction model, a gradient cut-off operation is performed on its output to freeze its gradient propagation, so as to ensure that the training of the second sampling reconstruction model will not affect the parameter update of the first sampling reconstruction model, thereby avoiding mutual interference between the two models during the optimization process. Secondly, during the training of the second sampling reconstruction model, the gradient propagates normally and the shared parameters are updated, so that it can perform refined correction based on the output of the first sampling reconstruction model.

[0058] Specifically, the parameter update of the second sampling reconstruction model is based on the refined correction loss, and the shared parameters are updated through the backpropagation algorithm. Since the output of the first sampling reconstruction model is frozen, the gradient of the second sampling reconstruction model will not be backpropagated to the first sampling reconstruction model, thus ensuring that the training processes of the two models are independent of each other while achieving efficient cooperation with parameter sharing. It can be understood that the parameters of the first reconstruction network are not updated, and only the second reconstruction network is updated. Since the parameters are shared between the two networks, when the parameters of the second reconstruction network are updated, the parameters of the first reconstruction network will also be updated simultaneously.

[0059] In an embodiment of the present application, the process of performing model parameter update operations based on the loss function includes: performing model parameter update operations on the second sampling reconstruction model based on the loss function.

[0060] In this embodiment, the loss function is used to measure the difference between the current original sampling data and the second full sampling image. The loss function includes, but is not limited to, mean squared error (MSE), structural similarity index (SSIM), or perceptual loss, etc. Among them, MSE quantifies the overall difference by calculating the squared difference between two data points; SSIM evaluates the similarity of images from three dimensions: brightness, contrast, and structure.

[0061] In this embodiment, the parameters of the second sampling reconstruction model are updated through the backpropagation algorithm. The loss function is used to measure the difference between the model output and the actual data, providing an optimization target for the optimization algorithm; the backpropagation algorithm determines the contribution of each parameter to the loss layer by layer by calculating the gradient of the loss function with respect to the model parameters, obtaining the gradient value. Then, an optimization algorithm is used to update the model parameters according to the gradient value. The optimization algorithm includes, but is not limited to, gradient descent method, Adam optimizer, etc.

[0062] Exemplarily, when the gradient descent method is adopted, the parameters are adjusted along the opposite direction of the gradient at a fixed learning rate; when the Adam optimizer is adopted, it combines momentum and adaptive learning rate mechanisms to dynamically adjust the parameter update step size, accelerating convergence and improving stability. By iteratively executing the above process, the model parameters are gradually optimized, and the loss value gradually decreases, finally enabling the second sampling reconstruction model to more accurately complete the refined correction task. This process ensures the efficient training of the model and improves the accuracy and robustness of the reconstruction results.

[0063] More preferably, in an embodiment of the present application, an adaptive loss weight mechanism is introduced to update the parameters of the second sampling reconstruction model. Specifically, the adaptive loss weight mechanism means dynamically adjusting the weight of the loss function according to the importance of different regions or features. Exemplarily, for key anatomical structure regions in an image, a higher weight is assigned to ensure higher reconstruction accuracy of the model in these regions. This mechanism can be implemented in a data-driven manner, for example, dynamically adjusting the weight based on regional features or user annotation information.

[0064] As Figures 3 to 4 described above, the method proposed based on the present application was verified on the fastMRI knee dataset, and the consistency of the distribution of training data and actual application data and its impact on the reconstruction performance were evaluated through t-SNE clustering analysis. Figure 3 shows the distribution of training data and actual inference data in the prior art. The results show that the distributions of the two are relatively scattered, indicating an obvious difference between the training data and the actual application data. This difference easily leads to sub-optimal reconstruction results when the model is applied in practice, limiting the generalization ability and reconstruction performance of the model. In contrast, Figure 4 shows the distribution of the simulation data and the actual application data proposed in the present application, where the simulation data refers to the simulated undersampled image data in the present application. The results show that the distributions of the two are basically the same, indicating that the distribution difference between the training data and the actual application data has been effectively reduced. This distribution consistency ensures the robustness of the model in actual application and improves the reliability of the reconstruction results.

[0065] In addition, the effects of different training strategies of the present method were verified on the 4-fold undersampled fastMRI knee dataset, with SSIM and PSNR as evaluation metrics. As shown in Table 1, the key metrics in the reconstruction effect evaluation of the Proposed method, Supervised (a supervised learning method), and SSDU (Self-Supervised Data Utilization) proposed in the present application are shown, including the Structural Similarity Index Measure (SSIM) and the Peak Signal-to-Noise Ratio (PSNR). Among them, the SSIM metric is used to measure the similarity between the reconstructed image and the original image, while the PSNR metric is used to reflect the degree of the signal quality of the image relative to the noise.

[0066] In terms of the SSIM metric, the performance of the method of this application is superior to that of SSDU, with a specific value of 0.9156 ± 0.0157. In contrast, the SSIM value of the SSDU method is 0.9110 ± 0.0141. In addition, the SSIM value of this application is close to that of the Supervised method (0.9332 ± 0.0139), indicating the potential for further improvement in image reconstruction quality. Secondly, the PSNR results further verify the advantages of this application, with a value of 34.72 ± 1.74, which is significantly higher than 33.27 ± 1.92 of the SSDU method, although still lower than 35.27 ± 2.10 of the Supervised method. This shows that in terms of reconstruction performance, this application is superior to the SSDU method.

[0067] SSDU Proposed Supervised SSIM 0.9110±0.0141 0.9156±0.0157 0.9332±0.0139 PSNR 33.27±1.92 34.72±1.74 35.27±2.10

[0068] Table 1: Verification results of the fastMRI knee dataset with 4-fold undersampling

[0069] Therefore, it can be understood that the method proposed in this application significantly improves the performance of the self-supervised reconstruction method, further narrowing the gap with the supervised reconstruction method. Specifically, compared with the method of training by dividing data subsets (such as SSDU) in the prior art, this method effectively improves the reconstruction quality through simulated data generation and distribution consistency optimization. This improvement verifies the effectiveness of this method and provides new ideas and more practical solutions for the application of self-supervised learning in medical image reconstruction.

[0070] To sum up, this application mainly addresses the need for deep learning network training without relying on complete acquisition data, and proposes a novel input-label construction method. Specifically, the actually acquired data is used as the label, and the simulated generated image is used as the network input. The simulation process is as follows: the actually acquired image is input into the network, and the output generated by the network is regarded as the reconstruction result of the fully sampled image, and then the output data is simulated for undersampling, and this undersampling follows the same distribution characteristics as the actually acquired data.

[0071] This application improves the learning quality of the model through the generated simulated undersampled data. The simulated data provides a large number of training samples, enabling the image reconstruction model network to better learn various possible image features. As the network parameters are continuously updated, the quality of the simulated data gradually improves, and its distribution gradually approaches the actually acquired data. This process effectively reduces the distribution difference between the training data and the actually used data, thereby improving the generalization ability and reconstruction performance of the model, and making the model show better effects in real application scenarios.

[0072] The mode of combining the first sampling reconstruction model and the second sampling reconstruction model proposed in this application is a cascade network (Unrolled Network). The cascade network of this application gradually optimizes the reconstruction results through a series of iterative steps, and each step includes a network layer and a data consistency layer. The data consistency layer is used to ensure that the generated image maintains a high degree of similarity with the actual data, while the network layer introduces additional constraints by introducing regularization terms to improve the reconstruction quality and suppress unnecessary noise. This alternating optimization strategy can take into account the authenticity and aesthetics of the image at the same time during the learning process, thereby significantly improving the reconstruction effect.

[0073] In particular, the cascade network in this application is used to gradually adjust and optimize the simulated under-sampled data in each iterative step to make it closer to the actual collected data. This gradual optimization process not only improves the accuracy of the model, but also enhances its adaptability and robustness in real application scenarios. In addition, the design of the cascade network allows flexibility in the selection of network structure. When designing the model, there is no need to emphasize a specific network structure, but the most suitable network architecture can be selected according to specific application requirements. Therefore, this application provides a scientific and reasonable implementation path for self-supervised learning, which effectively improves the technical effect of medical image reconstruction.

[0074] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. For example, the first sampling reconstruction model and the second sampling reconstruction model are only used to distinguish different sampling reconstruction models, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0075] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" represent examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0076] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (item)" or its similar expression refers to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0077] Figure 5 is a schematic block diagram of a self - supervised reconstruction device 500 for magnetic resonance images provided by the embodiments of the present application. As Figure 5 shown, the device includes a model initialization module 501 and a model training module 502.

[0078] Model initialization module 501: is used to obtain a plurality of original sampling data.

[0079] Model training module 502: is used to sequentially input each original sampling data into the first sampling reconstruction model to generate a first fully sampled image; perform a simulated undersampling operation on the first fully sampled image to generate simulated undersampled data; input the simulated undersampled data into the second sampling reconstruction model to generate a second fully sampled image; calculate the loss function of the current original sampling data and the second fully sampled image, and perform a model parameter update operation based on the loss function.

[0080] It should be understood that the specific processes of each module performing the above - mentioned corresponding steps have been described in detail in the above - mentioned method embodiments. For the sake of brevity, they will not be repeated here.

[0081] It should also be understood that the division of modules in the embodiments of the present application is schematic, merely a logical function division. In actual implementation, there can be other division methods. Additionally, in each embodiment of the present application, the functional modules can be integrated in one processor, can exist alone physically, or two or more modules can be integrated in one module. The above - integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0082] Figure 6 is a schematic block diagram of an electronic terminal provided by the embodiments of the present application. As Figure 6As shown, the electronic terminal includes: at least one processor 601, a memory 602, at least one network interface 603, and a user interface 605. Each component in the device is coupled together through a bus system 606. It can be understood that the bus system 606 is used to implement the connection and communication between these components. In addition to the data bus, the bus system 606 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 6 all kinds of buses are labeled as the bus system.

[0083] Among them, the user interface 605 may include a display, a keyboard, a mouse, a trackball, a click gun, a key, a button, a touchpad, or a touch screen, etc.

[0084] It can be understood that the memory 602 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM, Read Only Memory), a programmable read-only memory (PROM, Programmable Read-Only Memory), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM, Static Random Access Memory), synchronous static random access memory (SSRAM, Synchronous Static Random Access Memory). The memory described in the embodiments of the present application is intended to include but not limited to these and any other suitable categories of memories.

[0085] The memory 602 in the embodiments of the present application is used to store various categories of data to support the operation of the electronic terminal 600. Examples of these data include: any executable program for operating on the electronic terminal 600, such as an operating system 6021 and application programs 6022; the operating system 6021 contains various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application programs 6022 can include various application programs, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. Implementing the self-supervised reconstruction method of magnetic resonance images provided by the embodiments of the present application can be included in the application programs 6022.

[0086] The method disclosed in the embodiments of the present application above can be applied to the processor 601 or implemented by the processor 601. The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method can be completed by the integrated logic circuit in hardware in the processor 601 or instructions in software form. The above-mentioned processor 601 may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 601 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor 601 may be a microprocessor or any conventional processor, etc. Combining the steps of the accessory optimization method provided in the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, and this storage medium is located in the memory. The processor reads the information in the memory and combines its hardware to complete the steps of the foregoing method.

[0087] In an exemplary embodiment, the electronic terminal 600 may be an application-specific integrated circuit (ASIC, Application Specific Integrated Circuit), a DSP, a programmable logic device (PLD, ProgrammableLogic Device), or a complex programmable logic device (CPLD, Complex Programmable Logic Device) for executing the foregoing method.

[0088] According to the method provided in the embodiments of the present application, the present application also provides a computer program product, which includes: computer program code. When the computer program code runs on a computer, it causes the computer to execute the self-supervised reconstruction method of magnetic resonance images in any one of the above embodiments.

[0089] According to the method provided in the embodiments of the present application, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores program code. When the program code runs on a computer, it causes the computer to execute the self-supervised reconstruction method of magnetic resonance images in any one of the above embodiments.

[0090] As used in this specification, the terms "component", "module", "system", etc. are used to denote computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and the computing device can be components. One or more components can reside in a process and / or an execution thread, and a component can be located on one computer and / or distributed between two or more computers. In addition, these components can execute from various computer-readable media storing various data structures. A component can communicate, for example, by signals according to one or more data packets (e.g., data from two components interacting with another component among a local system, a distributed system, and / or a network, such as via the Internet interacting with other systems through signals) through local and / or remote processes.

[0091] Those of ordinary skill in the art will appreciate that the various illustrative logical blocks and steps described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether such functions are implemented in hardware or software depends upon the particular application and design constraints of the technical solution. Skilled artisans may implement the described functions in different ways for each particular application, but such implementation should not be regarded as exceeding the scope of this application.

[0092] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, apparatuses, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0093] In several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed with each other can be through some interfaces, and the indirect couplings or communication connections of apparatuses or units can be in electrical, mechanical, or other forms.

[0094] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0095] In addition, in each embodiment of this application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0096] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, the processes or functions according to the embodiments of this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a high-density digital video disc (Digital Video Disc, DVD)), or a semiconductor medium (for example, a solid-state disk (Solid State Disk, SSD), etc.).

[0097] If a function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs.

[0098] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0099] In summary, this application provides a self-supervised reconstruction method, device, medium, program product, and terminal for magnetic resonance images. Through a two-stage model design, the data reconstruction and optimization processes are sequentially executed. The original undersampled data is input into the first sampling reconstruction model to generate the first fully sampled image; subsequently, a simulated undersampling operation is performed on the first fully sampled image to generate simulated undersampled data; the simulated undersampled data is input into the second sampling reconstruction model to generate a high-quality second fully sampled image; a loss function is constructed based on the original sampling data and the second fully sampled image, and the model parameters are updated and optimized. This solves the problem of inconsistent data distributions in the training stage and the inference stage in the prior art, and at the same time improves the data utilization efficiency and the model generalization ability during the training process. Without relying on a complete image or two acquisition processes, efficient and accurate magnetic resonance image reconstruction is achieved, and significant effects are achieved in reducing artifacts and improving image quality. Therefore, this application effectively overcomes various disadvantages in the prior art and has high industrial utilization value.

[0100] The above embodiments are only illustrative of the principles and effects of this application, and are not used to limit this application. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or changes completed by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed by this application should still be covered by the claims of this application.

Claims

1. A self-supervised reconstruction method for magnetic resonance images, characterized in that, It includes: Obtain a plurality of original sampling data; Input each original sampling data into the first sampling reconstruction model in sequence to generate a first fully sampled image; Perform a simulated undersampling operation on the first fully sampled image to generate simulated undersampled data; Input the simulated undersampled data into the second sampling reconstruction model to generate a second fully sampled image; Calculate the loss function of the current original sampling data and the second fully sampled image, and perform a model parameter update operation based on the loss function.

2. The self-supervised reconstruction method of magnetic resonance images according to claim 1, wherein It includes: The first sampling reconstruction model and the second sampling reconstruction model share parameters.

3. The self-supervised reconstruction method of magnetic resonance images according to claim 2, wherein It includes: Before inputting the simulated undersampled data into the second sampling reconstruction model, the following steps are also performed: Replace the original sampling data with the corresponding position of the simulated undersampled data according to a preset rule.

4. The self-supervised reconstruction method of magnetic resonance images according to claim 1, characterized in that The process of performing a simulated undersampling operation on the first fully sampled image to generate simulated undersampled data includes: Perform a Fourier transform operation on the first fully sampled image in sequence and multiply it by a preset sampling mask to generate the simulated undersampled data; wherein, the preset sampling mask is consistent with the acquisition distribution of the original sampling data.

5. The self-supervised reconstruction method of magnetic resonance images according to claim 1, wherein The method includes: performing a gradient cutoff operation on the output of the first sampling reconstruction model.

6. The self-supervised reconstruction method of magnetic resonance images according to claim 5, characterized in that, The process of performing a model parameter update operation based on the loss function includes: performing a model parameter update operation on the second sampling reconstruction model based on the loss function.

7. A self-supervised reconstruction device for magnetic resonance images, characterized in that, It includes: Model initialization module: used to obtain a plurality of original sampling data; Model training module: used to input each original sampling data into the first sampling reconstruction model in sequence to generate a first fully sampled image; Perform a simulated undersampling operation on the first fully sampled image to generate simulated undersampled data; input the simulated undersampled data into the second sampling reconstruction model to generate a second fully sampled image; calculate the loss function of the current original sampling data and the second fully sampled image, and perform a model parameter update operation based on the loss function.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the self-supervised reconstruction method of the magnetic resonance image according to any one of claims 1 to 6.

9. A computer program product, characterized in that, The computer program product includes computer program code, and when the computer program code runs on a computer, it enables the computer to implement the self-supervised reconstruction method of the magnetic resonance image according to any one of claims 1 to 6.

10. An electronic terminal, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the self-supervised reconstruction method of the magnetic resonance image according to any one of claims 1 to 6.

Images