A magnetic resonance image enhancement method, apparatus, electronic device, and computer-readable storage medium based on structure injection.

Abstract

This application relates to the field of medical image processing, providing a method, apparatus, electronic device, and computer-readable storage medium for magnetic resonance imaging (MRI) image enhancement based on structure injection. This application constructs a two-stream network model with an asymmetric topology. It utilizes the structural pyramid of a guided flow network to extract multi-scale anatomical priors for auxiliary modalities. A recovery flow network based on an encoder structure is used to extract features for the target modality, obtaining target features. Through a guided simple gating mechanism, spatial adaptive modulation of the target features is performed to inject anatomical priors into the target features. Image reconstruction is then performed based on the modulated target features, outputting an enhanced target modal MRI image. This application accurately restores high-frequency boundaries and texture details while suppressing noise and eliminating artifacts, thus improving the diagnostic quality of MRI.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and more specifically to a method, apparatus, electronic device, and computer-readable storage medium for magnetic resonance image enhancement based on structural injection. Background Technology

[0002] Magnetic resonance imaging (MRI) is a non-invasive, radiation-free medical imaging technique with irreplaceable value in clinical disease diagnosis, preoperative planning, and efficacy evaluation. However, traditional MRI often faces several image quality problems in clinical applications: images from low-field equipment have a low signal-to-noise ratio; undersampling strategies used to increase scan speed introduce aliasing artifacts; and limited imaging time sometimes results in images with low spatial resolution. These factors severely affect the diagnostic usability of the images. Traditional methods to improve image quality usually rely on extending the scan time, but this not only reduces the equipment's throughput efficiency but may also introduce new motion artifacts due to patient movement, thus affecting the accuracy and reliability of the diagnosis.

[0003] In recent years, deep learning-based medical image processing methods have shown great potential in the field of image restoration. However, existing methods often have limitations when performing denoising, artifact removal, or super-resolution reconstruction on magnetic resonance images: on the one hand, these methods experience a sharp decline in performance when dealing with extreme degradation conditions such as extremely low signal-to-noise ratios, high-magnification undersampling, or large-scale super-resolution, as the limited information contained in a single modality image makes it difficult to support high-quality reconstruction; on the other hand, when attempting to restore images, they often over-smooth tissue edges or blur fine textures, leading to the loss of important anatomical structural features and affecting doctors' visual diagnosis and subsequent quantitative analysis.

[0004] It is noteworthy that in standard clinical MRI examination procedures, for comprehensive diagnosis, multiple contrast image sequences (such as T1-weighted, T2-weighted, and proton density-weighted images) are typically acquired in the same examination. This means that when acquiring the target modality image to be enhanced (such as an undersampled and aliased T2 image), its corresponding high-quality auxiliary modality image (such as a clear T1 image) can be naturally obtained as prior information, without requiring the patient to undergo additional scans or increasing examination time and costs. This provides a natural data foundation for image enhancement using cross-modal information.

[0005] However, existing cross-modal enhancement algorithms generally face two major challenges when fusing information from different modalities: First, structural mismatch between modalities. Due to the different imaging principles and contrast mechanisms of different sequences, the same anatomical structure may appear differently in images of different modalities, and direct fusion may lead to structural alignment errors. Second, leakage of contrast information. During the fusion process, the features of the auxiliary modality may inappropriately contaminate the target modality, causing the reconstructed image to lose its inherent contrast characteristics and generate invalid results with modal confusion.

[0006] Therefore, developing a magnetic resonance imaging enhancement method that can fully utilize high-quality auxiliary modal prior information and effectively achieve joint denoising, artifact suppression, and super-resolution reconstruction while strictly maintaining the inherent contrast of the target modality is of great practical significance for improving image quality, ensuring diagnostic accuracy, and promoting the clinical application of low-field and fast imaging technologies. It has become a key technical bottleneck in this field that urgently needs to be overcome. Summary of the Invention

[0007] In view of this, this application proposes a magnetic resonance image enhancement method, apparatus, electronic device and computer-readable storage medium based on structure injection, to overcome the problems of existing magnetic resonance image enhancement techniques in processing low signal-to-noise ratio, undersampled or low spatial resolution images, which are difficult to accurately preserve high-frequency details while suppressing noise and artifacts, and multimodal feature fusion is prone to leakage of contrast information, semantic abstraction and truncation of weak features by nonlinear activation functions.

[0008] Specifically, this application is implemented through the following technical solution:

[0009] According to a first aspect of the embodiments of this specification, a method, apparatus, and electronic device for magnetic resonance image enhancement based on structure injection are provided, the method comprising the following steps:

[0010] Step S1: Acquire the target modal magnetic resonance image to be processed and the auxiliary modal structural image used to provide structural reference at the same anatomical location;

[0011] Step S2: Input the target modal magnetic resonance image and the auxiliary modal structure image into a pre-trained two-stream network model to obtain the enhanced target modal magnetic resonance image; the two-stream network model includes a guiding flow network and a recovery flow network, wherein the guiding flow network and the recovery flow network form an asymmetric topology, wherein:

[0012] The guiding flow network adopts a pyramid structure to extract features from auxiliary modal structure images and obtain multi-scale anatomical structure prior features.

[0013] The recovery stream network employs an encoder and decoder structure to extract features from the target modal magnetic resonance image, obtain target features, and reconstruct the image based on the modulated target features, outputting an enhanced target modal magnetic resonance image. Only at each encoder layer stage, a guided simple gating mechanism is used to spatially adaptively modulate the target features using prior anatomical features of the corresponding scale, so as to inject the prior anatomical features into the target features, obtaining modulated target features.

[0014] According to a second aspect of the embodiments of this specification, a magnetic resonance image enhancement apparatus based on structure injection is provided, the apparatus comprising:

[0015] The image acquisition unit is used to acquire the target modal magnetic resonance image to be processed and the auxiliary modal structural image for providing a structural reference at the same anatomical location;

[0016] An image processing unit is configured to input the target modal magnetic resonance image and the auxiliary modal structure image into a pre-trained two-stream network model to obtain an enhanced target modal magnetic resonance image. The two-stream network model includes a guiding flow network and a recovery flow network, wherein the guiding flow network and the recovery flow network form an asymmetric topology.

[0017] The guiding flow network adopts a pyramid structure to extract features from auxiliary modal structure images and obtain multi-scale anatomical structure prior features.

[0018] The recovery stream network employs an encoder and decoder structure to extract features from the target modal magnetic resonance image, obtain target features, and reconstruct the image based on the modulated target features, outputting an enhanced target modal magnetic resonance image. Only at each encoder layer stage, a guided simple gating mechanism is used to spatially adaptively modulate the target features using prior anatomical features of the corresponding scale, so as to inject the prior anatomical features into the target features, obtaining modulated target features.

[0019] According to a third aspect of the embodiments of this specification, an electronic device is provided, comprising: a processor; and a computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in the first aspect.

[0020] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions thereon, characterized in that the instructions, when executed by a processor, implement the method as described in the first aspect.

[0021] The embodiments of this application have at least the following technical effects:

[0022] This application embodiment solves the modality-specific contrast contamination problem caused by traditional early fusion paradigms by constructing a dual-stream asymmetric topology architecture to explicitly decouple the anatomical structure information in the auxiliary modality from the contrast information in the target modality.

[0023] Meanwhile, this application proposes a Guided Simple Gating (GSG) mechanism, which uses extracted multi-scale anatomical structure prior features to predict spatial affine parameters and spatially modulate target features, thereby accurately and selectively recovering high-frequency anatomical structure details while filtering out severe noise, eliminating undersampling artifacts, and improving spatial resolution.

[0024] In summary, the embodiments of this application can output high-quality images with excellent anatomical fidelity and contrast accuracy in various clinical magnetic resonance imaging scenarios such as image denoising, artifact removal, and super-resolution reconstruction. Attached Figure Description

[0025] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0026] Figure 1 This is a schematic flowchart illustrating an exemplary embodiment of a magnetic resonance image enhancement method based on structure injection.

[0027] Figure 2 This is a schematic diagram of a network architecture for a two-stream network model illustrated in an exemplary embodiment of this application;

[0028] Figure 3 This is a schematic diagram of the internal processing flow of a guided simple gating mechanism shown in an exemplary embodiment of this application;

[0029] Figure 4 This is a schematic diagram illustrating the training process of a two-stream network model according to an exemplary embodiment of this application;

[0030] Figure 5 This is a schematic diagram illustrating the enhancement effect of a target modal magnetic resonance image according to an exemplary embodiment of this application;

[0031] Figure 6 This is a schematic diagram illustrating the effect of image enhancement based on a conventional method, as shown in an exemplary embodiment of this application.

[0032] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment of this application;

[0033] Figure 8 This is a block diagram illustrating a structure-injection-based magnetic resonance image enhancement device according to an exemplary embodiment of this application. Detailed Implementation

[0034] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0035] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0036] The embodiments described in this specification will now be described in detail.

[0037] This application provides a magnetic resonance image enhancement method based on structure injection. This magnetic resonance image enhancement method can be executed by an image processing system, which can be deployed on an electronic device or in the cloud.

[0038] Figure 1 This is a schematic flowchart illustrating an exemplary embodiment of a magnetic resonance image enhancement method based on structure injection, as shown in this application. Figure 1 As shown, the magnetic resonance image enhancement method includes the following steps:

[0039] Step S1: Acquire the target modal magnetic resonance image to be processed and the auxiliary modal structural image used to provide a structural reference at the same anatomical location.

[0040] The image quality of the auxiliary modal structure image is higher than that of the target modal magnetic resonance image. The image quality includes at least one of signal-to-noise ratio, aliasing artifact degree, or spatial resolution. That is, the target modal magnetic resonance image is a low-quality magnetic resonance image to be enhanced, including but not limited to low signal-to-noise ratio images, undersampled aliased images, and low spatial resolution images. The low signal-to-noise ratio image refers to an image acquired by a low-field magnetic resonance device with a main magnetic field strength of no more than 0.5T, or a mode with an inherently low signal-to-noise ratio. The undersampled aliased image refers to an image reconstructed from k-space data acquired using an undersampling strategy (undersampling factor of 2 to 8 times) to accelerate scanning, and the image contains aliasing artifacts. The low spatial resolution image refers to an image with blurred details due to limited scanning matrix (e.g., 128×128), thick slice (e.g., more than 5mm), or insufficient acquisition of high-frequency information in k-space.

[0041] The auxiliary modal structural image is a high-quality image with a different contrast obtained at the same anatomical location and within the same examination time as the target modal magnetic resonance image. The target modal magnetic resonance image and the auxiliary modal structural image have different magnetic resonance contrast types; for example, the target modal is a T2-weighted image and the auxiliary modal is a T1-weighted image; the target modal is a T2-weighted image and the auxiliary modal is a proton density-weighted image, etc.

[0042] In routine clinical examinations, multiple contrast sequences are typically acquired in a single MRI scan. Therefore, auxiliary modal structural images can be obtained naturally without additional scans, without increasing the patient's examination time or cost. For example, in brain MRI scans, T1-weighted images typically have a high signal-to-noise ratio and clear anatomical boundaries, and can be used as auxiliary modal structural images to enhance low-quality T2-weighted images.

[0043] In this embodiment, the target modal magnetic resonance image and the auxiliary modal structural image need to undergo spatial alignment preprocessing to ensure that they correspond at the pixel level in anatomical position. The alignment method can be rigid body registration or affine registration, or since the two sequences were acquired in the same examination and the patient did not move, the original DICOM coordinate information can be used directly for position matching.

[0044] Step S2: Input the target modal magnetic resonance image and the auxiliary modal structure image into a pre-trained two-stream network model to obtain the enhanced target modal magnetic resonance image; the two-stream network model includes a guiding flow network and a recovery flow network, wherein the guiding flow network and the recovery flow network form an asymmetric topology, wherein:

[0045] The guiding flow network adopts a pyramid structure to extract features from auxiliary modal structure images and obtain multi-scale anatomical structure prior features.

[0046] The recovery stream network employs an encoder and decoder structure to extract features from the target modal magnetic resonance image, obtain target features, and reconstruct the image based on the modulated target features, outputting an enhanced target modal magnetic resonance image. Only at each encoder layer stage, a guided simple gating mechanism is used to spatially adaptively modulate the target features using prior anatomical features of the corresponding scale, so as to inject the prior anatomical features into the target features, obtaining modulated target features.

[0047] Depend on Figure 1 As shown in the magnetic resonance image enhancement method, this implementation abandons the traditional single-modal mapping and simple feature fusion approach. By constructing an asymmetric two-stream network model, it achieves effective decoupling between modality-specific contrast and shared anatomical structural information. This embodiment can preserve the contrast characteristics of the target modality itself while suppressing extremely high levels of noise and aliasing artifacts, achieving accurate restoration of high-frequency textures.

[0048] In some embodiments, the pyramid structure of the guiding flow network includes multiple feature extraction levels with different resolutions, and each feature extraction level employs a parameterless gated activation function.

[0049] This embodiment employs a lightweight pyramid structure, which consists of multiple levels with different resolutions. For example... Figure 2 As shown, the guided flow network employs a four-layer pyramid structure. In each feature extraction layer, traditional nonlinear activation functions (such as ReLU, sigmoid, etc.) are abandoned, and a parameterless gated activation function is used for nonlinear transformation. This parameterless gated activation function does not contain learnable parameters and achieves feature selection only through simple gating logic, such as binary or soft gating based on the positive or negative value or magnitude of the input value. This effectively extracts anatomical structural information while avoiding semantic abstraction and modality-specific contrast leakage pollution.

[0050] As an example implementation, each feature extraction layer employs a 3×3 convolution with a stride of 2 for downsampling to progressively reduce the spatial resolution of the feature maps and expand the receptive field. The downsampled feature maps are then input into a parameterless gated activation function to obtain the output features of that layer. Through the stacking of multiple layers, the guiding flow network sequentially outputs prior anatomical features at multiple scales, from high to low resolution, for the subsequent recovery flow network to perform spatial adaptive modulation at different encoding stages.

[0051] Those skilled in the art should understand that the downsampling operation is not limited to a 3×3 convolution with a stride of 2. The kernel size and stride can also be adjusted according to actual needs. For example, a 5×5 convolution or a convolution with a stride of 1 can be used in conjunction with a pooling layer to achieve downsampling.

[0052] Through the aforementioned lightweight pyramid structure, the guiding flow network extracts multi-scale, pure anatomical prior information with extremely low computational overhead, providing high-quality structural guidance for subsequent injection operations.

[0053] The recovery stream network is a four-layer symmetric encoder-decoder structure built on a nonlinear activated free network, with feature fusion nodes positioned between the guiding stream pyramid and the corresponding layers of the recovery stream encoder. This asymmetric topology design enables the extraction of auxiliary structures with extremely low computational overhead, while allocating computational resources to the signal recovery of the primary target.

[0054] In some embodiments, the recovery stream network is used for feature extraction and image reconstruction of target modal magnetic resonance images. The recovery stream network adopts an encoder-decoder structure and is constructed based on nonlinear activation free blocks to avoid the truncation effect of traditional nonlinear activation functions (such as ReLU) on weak high-frequency features, thereby accurately preserving tissue boundaries and texture details while performing denoising, artifact removal, or super-resolution reconstruction.

[0055] In some possible implementations of this embodiment, each encoder and decoder layer of the recovery stream network contains multiple stacked nonlinear activation free blocks. Each nonlinear activation free block implements a nonlinear transformation through a gated linear unit (GLU) or element-wise multiplication logic. Each nonlinear activation free block does not contain a traditional nonlinear activation function (e.g., ReLU, sigmoid, tanh, etc.), but instead implements a nonlinear transformation through a gated linear unit (GLU) or element-wise multiplication logic.

[0056] For example, each nonlinear activation free block can contain two branches: one branch generates a nonlinear gated signal through a convolutional layer and GLU, and the other branch preserves the original features or performs a linear transformation. Finally, the results of the two branches are multiplied element-wise, thereby preserving weak signal components while introducing nonlinearity.

[0057] In the encoder stage, the recovery flow network extracts features from the target modal magnetic resonance image layer by layer, obtaining target features at different resolutions. In each encoder layer, a guided simple gating mechanism is used to spatially adaptively modulate the target features using prior anatomical features at the corresponding scale, injecting these prior anatomical features into the target features. This modulation process is performed only in the encoder stage; the decoder stage no longer receives prior anatomical features from the guided flow network. This ensures that the decoder can reconstruct the image based on pure target features, avoiding contrast contamination from auxiliary modalities.

[0058] In the decoder stage, the recovery stream network gradually recovers the spatial resolution based on the modulated target features through upsampling operations (such as transposed convolution or interpolation combined with convolution), and makes skip connections with the features of the corresponding level in the encoder, finally outputting an enhanced target modal magnetic resonance image. Since the decoder stage does not introduce auxiliary modal information, the reconstructed image can strictly maintain the inherent contrast characteristics of the target modality.

[0059] like Figure 2 As shown, the recovery stream network is a four-layer symmetrical encoder-decoder structure. The four layers of the encoder correspond to the four output scales of the guided stream network pyramid structure. In each encoder layer, a guided simple gating mechanism is used to spatially adaptively modulate the target features of the current layer; that is, the anatomical prior features of the corresponding scale output by the guided stream network are used to modulate the target features extracted by the recovery stream encoder. The four layers of the decoder employ convolution and upsampling operations symmetrical to the encoder, but no further cross-modal feature modulation is performed. This asymmetric injection topology tilts computational resources towards main signal recovery while effectively decoupling anatomical information from modality-specific contrast information.

[0060] In some embodiments, the Guided Simple Gating (GSG) mechanism is used to inject the anatomical prior features extracted by the guided flow network into the target features of the recovery flow network. See also Figure 3 The specific processing flow of this mechanism includes the following steps:

[0061] First, predict the spatial affine parameters.

[0062] The prior anatomical features output by the guiding flow network at various scales are processed by convolution. For example, a 3×3 convolution is used to map the prior anatomical features to spatial affine parameters, which include scaling parameters and offset parameters.

[0063] Then, the target features are segmented.

[0064] The target features of the current encoder level of the recovery stream network are segmented along the channel dimension into first features and second features. At this time, the segmented first features and second features are completely symmetrical. The segmentation method can be slicing by channel index, for example, taking the first half of the channel as the first feature and the second half of the channel as the second feature.

[0065] Finally, radiometric transformation and fusion.

[0066] The current target feature is segmented along the channel dimension into a first feature and a second feature. An affine transformation is performed on the first feature using the spatial affine parameters. The transformed first feature is then multiplied element-wise with the second feature to obtain the modulated target feature. Specifically, the first feature is multiplied by the scaling parameter and then added to the offset parameter to obtain the affine transformation result of the first feature. The affine transformation result of the first feature is then multiplied element-wise with the second feature, thereby completing the smooth cross-modal injection of the high-frequency anatomical structure.

[0067] During model training, the unmodulated second feature learning retains the original, general basic feature information of the target modality, while the modulated first feature learning is more susceptible to the influence of the auxiliary modality on features such as edges or textures.

[0068] Through the above network architecture, the recovery stream network can recover high-quality magnetic resonance images with clear anatomical structures and true contrast under extreme degradation conditions such as high magnification undersampling, extremely low signal-to-noise ratio, and large-scale super-resolution.

[0069] It is worth noting that although this embodiment takes noise reduction in a low-field magnetic resonance scenario as a specific case, it is also applicable to undersampling acceleration imaging at any magnification in high-field or low-field magnetic resonance, super-resolution reconstruction at different magnifications, and image quality enhancement in complex environments due to its overcoming of nonlinear activation cutoff effect and suppression of cross-modal contrast contamination.

[0070] After constructing the dual-stream network model of this application through the above embodiments, through Figure 4 The steps shown are used to train the two-stream network model.

[0071] First, the dataset is constructed.

[0072] The training dataset in this embodiment includes the target modal magnetic resonance image to be enhanced and the auxiliary modal structure image spatially aligned with it. The target modal magnetic resonance image to be enhanced includes low signal-to-noise ratio images, aliased images obtained by undersampling scans, or low spatial resolution images. The auxiliary modal structure image is a high-quality image used to provide a reference for the same anatomical location.

[0073] Next, we design the loss function.

[0074] Image samples from the training dataset are input into the two-stream network model to obtain enhanced target modal MRI sample images. A joint loss function is calculated based on these enhanced images and the label images. The weight parameters in the two-stream network model are continuously optimized using backpropagation until the joint loss function value decreases to a preset threshold or a preset number of training epochs is reached. Upon reaching the stopping condition, the training loop is terminated, and the parameters of the network model that performs best on the validation set are archived, resulting in the trained two-stream network model. The number of training epochs is typically set to the optimal number of training iterations observed on the validation data to prevent overfitting to specific training samples, thereby ensuring generalization ability on unknown clinical data.

[0075] The joint loss function The expression is as follows:

[0076] (1)

[0077] in, , and These represent Charbonnier loss, spatial gradient loss, and Fourier domain loss, respectively. , and These are the corresponding weighting coefficients. For example, =1.0, =0.03, =0.1.

[0078] The Charbonnier loss Used to enhance the consistency between an image and the real image at the pixel level, its expression is:

[0079] (2)

[0080] in, For the enhanced target modal magnetic resonance sample image, For the corresponding label image, The smoothing constant is set to a value of in this embodiment. .

[0081] The spatial gradient loss The expression used to constrain the sharpness of image edges and structures is:

[0082] (3)

[0083] in, and These represent the gradient operators in the horizontal and vertical directions, respectively. By minimizing the first-order norm distance between the enhanced target modal magnetic resonance sample image and the label image in the gradient domain, the clarity of the anatomical boundary is improved.

[0084] The Fourier domain loss It is used to compensate for high-frequency texture details in the frequency domain, and its expression is:

[0085] (4)

[0086] in, This refers to the Fast Fourier Transform, which improves the network's ability to recover high-frequency features by constraining the consistency of the frequency distribution of the image in the transform domain.

[0087] To further illustrate the technical effects of the present application's technical solution, this embodiment verifies the present application's technical solution on a machine equipped with an AMD Ryzen 9 5900HX central processing unit, an NVIDIA RTX3060 graphics processor, and 6GB of memory, based on the publicly available M4RAW dataset.

[0088] Figure 5 This diagram illustrates the effect of image enhancement using the dual-stream network model of this application. (Refer to...) Figure 5 Sub-image (a) in the image is the original input image, which is a target modal magnetic resonance image affected by noise. Figure 5 Subimage (b) is a magnified view of the marked region in subimage (a), showing blurred tissue boundaries and severe loss of high-frequency texture details. After enhancement using the dual-stream network model of this application, the enhanced result of the original input image is obtained, as shown below. Figure 5 The subgraph (c) is shown. Figure 5 Sub-image (d) is an enlarged view of the marked area in sub-image (c). Combining sub-image (c) and sub-image (d), it can be seen that the dual-stream network model of this application effectively removes image noise, successfully restores clear anatomical structure boundaries and high-frequency details, and perfectly preserves the unique contrast of the target modal magnetic resonance image.

[0089] Figure 6 This diagram illustrates the effect of image enhancement using traditional methods. This embodiment... Figure 5 The original input image is linearly interpolated to obtain the enhanced result of the original input image, such as... Figure 6 The subgraph (a) is shown; Figure 6 Sub-image (b) is a magnified view of the marked area in sub-image (a). The comparison shows that traditional methods can only smooth the transition of pixels, resulting in obvious artifacts and blurring of the overall image, and are unable to reconstruct the true pathological or anatomical high-frequency features.

[0090] In summary, this application constructs a two-stream asymmetric topology architecture to explicitly decouple the anatomical information in the auxiliary modality from the contrast information in the target modality, thus solving the modality-specific contrast contamination problem caused by traditional early fusion paradigms. Simultaneously, this application proposes a guided gating mechanism that uses extracted multi-scale prior features to predict spatial affine parameters and spatially modulate the target features, thereby precisely and selectively recovering high-frequency anatomical details while filtering out severe noise, eliminating undersampling artifacts, and improving spatial resolution. Furthermore, the nonlinear activation-free architecture employed in the recovery stream network replaces conventional activation functions with element-wise multiplication operations, effectively avoiding the risk of truncating weak detail features and addressing the problem of deep semantic abstraction.

[0091] This application effectively bridges the gap between fidelity and perceived quality in enhanced images by fully exploring the auxiliary modal information acquired during clinical examinations without requiring additional scanning, without increasing the scanning burden on patients or the workload of clinical staff, and by combining the constraints of multi-domain hybrid loss functions such as multi-spatial gradient loss and Fourier domain loss.

[0092] This application can output high-quality images with excellent anatomical fidelity and contrast accuracy in various clinical magnetic resonance imaging scenarios such as image denoising, artifact removal, and super-resolution reconstruction.

[0093] Figure 7 This is a schematic diagram of an electronic device illustrated in this specification according to an exemplary embodiment. Please refer to... Figure 7 At the hardware level, the device includes a processor 710, an internal bus 720, a network interface 730, memory 740, a hardware acceleration device 750, and non-volatile memory 760, and may also include other hardware required for its functions. One or more embodiments of this application can be implemented in software, for example, the processor 710 reads the corresponding computer program from the non-volatile memory 760 into the memory 740 and then runs it. Of course, in addition to software implementation, one or more embodiments of this application do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0094] Figure 8 This is a structural block diagram illustrating an exemplary embodiment of a magnetic resonance image enhancement device based on structure injection, which can be applied to, for example... Figure 8 The electronic device shown implements the technical solution of this application. The magnetic resonance image enhancement device includes: an image acquisition unit 810 and an image processing unit 820, wherein:

[0095] The image acquisition unit 810 is used to acquire the target modal magnetic resonance image to be processed and the auxiliary modal structural image for providing a structural reference at the same anatomical location;

[0096] Image processing unit 820 is used to input the target modal magnetic resonance image and the auxiliary modal structure image into a pre-trained two-stream network model to obtain an enhanced target modal magnetic resonance image; the two-stream network model includes a guiding flow network and a recovery flow network, wherein the guiding flow network and the recovery flow network form an asymmetric topology, wherein:

[0097] The guiding flow network adopts a pyramid structure to extract features from auxiliary modal structure images and obtain multi-scale anatomical structure prior features.

[0098] The recovery stream network employs an encoder and decoder structure to extract features from the target modal magnetic resonance image, obtain target features, and reconstruct the image based on the modulated target features, outputting an enhanced target modal magnetic resonance image. Only at each encoder layer stage, a guided simple gating mechanism is used to spatially adaptively modulate the target features using prior anatomical features of the corresponding scale, so as to inject the prior anatomical features into the target features, obtaining modulated target features.

[0099] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0100] Accordingly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the above embodiments.

[0101] Accordingly, embodiments of this application also provide a computer program product configured to perform the methods described in any of the above embodiments.

[0102] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0103] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0104] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0105] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0106] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0107] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0108] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0109] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0110] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A magnetic resonance image enhancement method based on structure injection, characterized in that, The method includes the following steps: Step S1: Acquire the target modal magnetic resonance image to be processed and the auxiliary modal structural image used to provide structural reference at the same anatomical location; Step S2: Input the target modal magnetic resonance image and the auxiliary modal structure image into a pre-trained two-stream network model to obtain the enhanced target modal magnetic resonance image; the two-stream network model includes a guiding flow network and a recovery flow network, wherein the guiding flow network and the recovery flow network form an asymmetric topology, wherein: The guiding flow network adopts a pyramid structure to extract features from auxiliary modal structure images and obtain multi-scale anatomical structure prior features. The recovery stream network employs an encoder and decoder structure to extract features from the target modal magnetic resonance image, obtain target features, and reconstruct the image based on the modulated target features, outputting an enhanced target modal magnetic resonance image. Only at each encoder layer stage, a guided simple gating mechanism is used to spatially adaptively modulate the target features using prior anatomical features of the corresponding scale, so as to inject the prior anatomical features into the target features, obtaining modulated target features.

2. The method according to claim 1, characterized in that, The pyramid structure of the guiding flow network includes multiple feature extraction levels with different resolutions, and each feature extraction level uses a parameterless gated activation function.

3. The method according to claim 1, characterized in that, The method of spatially adaptively modulating the target features using prior anatomical features at a corresponding scale through a guided simple gating mechanism includes: The prior features of the anatomical structure at the current scale are convolved to obtain spatial affine parameters. The current target feature is divided into a first feature and a second feature along the channel dimension. The first feature is then subjected to an affine transformation using the spatial affine parameters. The first feature after the affine transformation is multiplied element-wise with the second feature to obtain the modulated target feature.

4. The method according to claim 3, characterized in that, The spatial affine parameters include scaling parameters and offset parameters, and the affine transformation of the first feature using the spatial affine parameters includes: Multiplying the first feature by the scaling parameter and then adding the offset parameter yields the affine transformation result of the first feature.

5. The method according to claim 1, characterized in that, Each encoder and decoder layer of the recovery stream network contains multiple stacked nonlinear activation free blocks, and each nonlinear activation free block implements nonlinear transformation through gated linear units or element-wise multiplication logic.

6. The method according to any one of claims 1 to 5, characterized in that: The target modal magnetic resonance image and the auxiliary modal structural image have different magnetic resonance contrast types; Alternatively, the image quality of the auxiliary modal structure image is higher than that of the target modal magnetic resonance image, and the image quality includes at least one of signal-to-noise ratio, degree of aliasing artifacts, or spatial resolution.

7. A magnetic resonance image enhancement device based on structure injection, characterized in that, The device includes: The image acquisition unit is used to acquire the target modal magnetic resonance image to be processed and the auxiliary modal structural image for providing a structural reference at the same anatomical location; An image processing unit is configured to input the target modal magnetic resonance image and the auxiliary modal structure image into a pre-trained two-stream network model to obtain an enhanced target modal magnetic resonance image. The two-stream network model includes a guiding flow network and a recovery flow network, wherein the guiding flow network and the recovery flow network form an asymmetric topology. The guiding flow network adopts a pyramid structure to extract features from auxiliary modal structure images and obtain multi-scale anatomical structure prior features. The recovery stream network employs an encoder and decoder structure to extract features from the target modal magnetic resonance image, obtain target features, and reconstruct the image based on the modulated target features, outputting an enhanced target modal magnetic resonance image. Only at each encoder layer stage, a guided simple gating mechanism is used to spatially adaptively modulate the target features using prior anatomical features of the corresponding scale, so as to inject the prior anatomical features into the target features, obtaining modulated target features.

8. An electronic device, characterized in that, include: processor; A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the method as described in any one of claims 1 to 6.

Images