Magnetic Resonance Image Synthesis Method Based on Modality Adaptation and Multi-Scale Attention

Abstract

This invention relates to the field of image processing and provides a magnetic resonance image synthesis method based on modal adaptation and multi-scale attention. The method includes: obtaining class probabilities from a target image; constructing modal feature units using a discrete sampling strategy; obtaining modal memory representations and constructing modal memory units; constructing point-level branches, local branches, mid-range branches, and global branches, obtaining point-level features through point-level branches, local features through local branches, mid-range features through mid-range branches, and global features through global branches; constructing multi-scale feature units, using modal feature units, modal memory units, and multiple multi-scale feature units to form a feature modeling unit; inputting a training image into the feature modeling unit, calculating the generator's total loss for the generated image, training the feature modeling unit based on the generator's total loss, and using the trained feature modeling unit to obtain the target synthesized image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a magnetic resonance image synthesis method based on modal adaptation and multi-scale attention. Background Technology

[0002] Multiphase contrast-enhanced magnetic resonance imaging (MRI) of the liver plays a crucial role in the screening, diagnosis, classification, and efficacy evaluation of liver diseases such as hepatocellular carcinoma. The arterial phase, reflecting tumor blood supply characteristics and vascular enhancement patterns, is one of the most critical phases in clinical interpretation. However, the arterial phase acquisition window is narrow and easily affected by respiratory coordination, scanning timing errors, and individual hemodynamic differences, leading to problems such as missing images, insufficient enhancement, or severe artifacts in clinical arterial phase images. Furthermore, arterial phase imaging relies on contrast agent injection, and its use is limited by factors such as patient renal function, allergic reactions, and potential safety risks. Therefore, synthesizing high-quality arterial phase images from non-contrast images without increasing the burden of contrast agent use is of great significance.

[0003] The clinical arterial phase can often be further subdivided into early and late arterial phases, which differ in liver parenchyma, vascular enhancement morphology, and lesion enhancement patterns. Because publicly available data or clinical archives often treat the arterial phase as a single category, training data with similar labels actually mix different sub-phases, resulting in multiple "reasonable" arterial phase outputs corresponding to the same non-enhanced input. If a single deterministic generator is used to directly regress the target image, the model tends to learn an averaging mapping, manifested as weakened vascular contrast, "flattened" lesion enhancement morphology, and blurred boundaries and textures.

[0004] Furthermore, magnetic resonance imaging often contains multi-scale information, including organ-level contours, intrahepatic vascular trees, small vessel branches, and heterogeneous textures within lesions. Relying solely on local receptive fields based on convolution often fails to model vascular continuity and cross-regional structural consistency; while directly using global self-attention incurs high computational overhead and is difficult to train stably at high resolutions. Summary of the Invention

[0005] This invention aims to at least solve one of the technical problems existing in related technologies. To this end, this invention provides a magnetic resonance image synthesis method based on modal adaptation and multi-scale attention, which achieves the synthesis of high-quality arterial phase images.

[0006] This invention provides a magnetic resonance image synthesis method based on modal adaptation and multi-scale attention, comprising: S1: Obtain the target image, obtain the category probability based on the target image, and adopt a discrete sampling strategy for the category probability to complete the construction of the modal feature unit; S2: Obtain the memory matrix, and complete the construction of modal memory units through the memory matrix and modal feature selection; S3: Construct point-level branches, local branches, mid-range branches, and global branches. Point-level branches highlight details to obtain point-level features, local branches enhance local context to obtain local features, mid-range branches strengthen structural expression to obtain mid-range features, and global branches capture long-range dependencies to obtain global features. S4: Construct multi-scale feature units through point-level features, local features, mid-range features, and global features, and use modal feature units, modal memory units, and multiple multi-scale feature units to form feature modeling units; S5: Obtain the training image, input the training image into the feature modeling unit to obtain the generated image, calculate the generator total loss of the generated image, train the feature modeling unit based on the generator total loss, and use the trained feature modeling unit to obtain the target synthesized image.

[0007] According to the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention provided by the present invention, step S1 further includes: S11: Obtain the target image, perform category probability determination on the target image, and obtain the category probability; S12: Perform discrete distribution sampling and one-hot encoding on the category probabilities to complete the construction of the modal memory unit.

[0008] According to the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention provided by the present invention, in step S2, a memory matrix is ​​obtained, and the memory matrix is ​​extracted through the modal feature selection to obtain a modal memory representation, thereby completing the construction of the modal memory unit.

[0009] According to the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention provided by the present invention, in step S3, point-level input features are obtained, and small-scale nonlinear mapping and activation are performed on the point-level input features to obtain the point-level features; Local input features are obtained, and depthwise separable convolution is performed on the local input features to obtain the local features.

[0010] According to the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention provided by the present invention, in step S3, mid-range input features are obtained, and the mid-range input features are modeled through a sparse attention mechanism to obtain the mid-range features; Obtain global input features, perform feature extraction and context aggregation on the global input features to obtain the global features.

[0011] According to the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention provided by the present invention, in step S4, multiple multi-scale feature units are obtained, and the multi-scale feature units are connected in series. The modal feature units and modal memory units are connected in series sequentially, and the series-connected modal feature units and modal memory units are connected in parallel to the series-connected multi-scale feature units to obtain the feature modeling unit.

[0012] This invention also provides a magnetic resonance image synthesis system based on modal adaptation and multi-scale attention, comprising: Modal Feature Unit Module: Used to acquire target images, obtain class probabilities based on target images, and adopt a discrete sampling strategy for class probabilities to complete the construction of modal feature units; Modal memory unit module: used to obtain the memory matrix, and complete the construction of the modal memory unit by selecting modal features through the memory matrix; Multi-scale feature module: used to construct point-level branches, local branches, mid-range branches and global branches. Point-level branches highlight details to obtain point-level features, local branches enhance local context to obtain local features, mid-range branches strengthen structural expression to obtain mid-range features, and global branches capture long-range dependencies to obtain global features. Feature modeling unit module: used to construct multi-scale feature units through point-level features, local features, mid-range features and global features, and to construct feature modeling units using modal feature units, modal memory units and multiple multi-scale feature units; The target image synthesis module is used to acquire training images, input the training images into the feature modeling unit to obtain generated images, calculate the total generator loss of the generated images, train the feature modeling unit based on the total generator loss, and use the trained feature modeling unit to obtain the target image synthesis.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention as described above.

[0015] The present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, which, when executed by a computer, enable the computer to perform the steps of any of the above-described methods for magnetic resonance image synthesis based on modal adaptation and multi-scale attention.

[0016] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: The magnetic resonance image synthesis method based on modal adaptation and multi-scale attention provided in this invention, through modal feature units and modal memory units, eliminates the need for additional manual strong annotation of sub-stages and does not heavily rely on prior rules. It can generate multiple candidate results by traversing the memory vector, explicitly expressing uncertainty and reducing the interference of sub-stage confounding on the regression target. The feature modeling unit can preserve pixel-level boundaries and textures, maintain local consistency, model mid-range structures, and capture global context with less computational resources.

[0017] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart of the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention provided by the present invention.

[0020] Figure 2 This is a schematic diagram of the magnetic resonance image synthesis system based on modal adaptation and multi-scale attention provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the magnetic resonance image synthesis device based on modal adaptation and multi-scale attention provided by the present invention.

[0022] Figure label: 100. Modal Feature Unit Module; 200. Modal Memory Unit Module; 300. Multi-Scale Feature Module; 400. Feature Modeling Unit Module; 500. Target Synthesis Image Module; 810. Processor; 820. Communication Interface; 830. Memory; 840. Communication Bus. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but cannot be used to limit the scope of this invention.

[0024] In the description of the embodiments of the present invention, it should be noted that the terms "first", "second" and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0025] In the description of the embodiments of the present invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of the present invention based on the specific circumstances.

[0026] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0027] The following is combined Figures 1 to 3 Specific embodiments of the present invention are described below. Figure 1 This is a flowchart illustrating the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention provided by the present invention, including: S1: Obtain the target image, obtain the category probability based on the target image, and adopt a discrete sampling strategy for the category probability to complete the construction of the modal feature unit; Furthermore, the objective of this stage is to obtain the category probabilities, thereby adopting a discrete sampling strategy to construct modal feature units. Specifically, step S1 further includes: S11: Obtain the target image, perform category probability determination on the target image, and obtain the category probability; S12: Perform discrete distribution sampling and one-hot encoding on the category probabilities to complete the construction of the modal memory unit.

[0028] The specific implementation method for the above steps in this embodiment is as follows: First, an enhanced arterial phase image needs to be obtained as the target image Y. Then, the target image is input into the output class probability model to determine its sub-phase category and perform class probability determination, thereby obtaining the class probability p. in, For outputting the class probability model, softmax() represents the softmax function.

[0029] Next, discrete distribution sampling and one-hot encoding are performed on the class probabilities to obtain modality feature selection. : in, To use the Gumbel-Softmax function for discrete sampling, For a pre-set temperature coefficient, The intermediate selection result is defined by `sg`, which is the stop-gradient operator, `arg max()`, which represents the `argmax` function, and `one-hot()`, which represents one-hot encoding. This completes the construction of the modal memory unit, allowing the network to form stable sub-stage partitions and corresponding feature decoupling internally.

[0030] S2: Obtain the memory matrix, and complete the construction of modal memory units through the memory matrix and modal feature selection; Furthermore, the objective of this stage is to obtain the modal memory representation, thereby completing the construction of the modal memory unit. Specifically, in step S2, a memory matrix is ​​obtained, and the memory matrix is ​​extracted through the modal feature selection to obtain the modal memory representation, thereby completing the construction of the modal memory unit.

[0031] The specific implementation method for the above steps in this embodiment is as follows: Here, we first need to obtain the pre-constructed memory matrix M, and then extract the memory matrix through modality feature selection to obtain the memory representation m corresponding to the target image: Where T represents transpose.

[0032] S3: Construct point-level branches, local branches, mid-range branches, and global branches. Point-level branches highlight details to obtain point-level features, local branches enhance local context to obtain local features, mid-range branches strengthen structural expression to obtain mid-range features, and global branches capture long-range dependencies to obtain global features. Furthermore, the objective of this stage is to obtain point-level features, local features, mid-range features, and global features. Specifically, in step S3, point-level input features are obtained, and small-scale nonlinear mapping and activation are performed on the point-level input features to obtain the point-level features; Local input features are obtained, and depthwise separable convolution is performed on the local input features to obtain the local features.

[0033] In step S3, the mid-range input features are obtained, and the mid-range input features are modeled using a sparse attention mechanism to obtain the mid-range features; Obtain global input features, perform feature extraction and context aggregation on the global input features to obtain the global features.

[0034] The specific implementation method for the above steps in this embodiment is as follows: First, the input features Z input to the multi-scale feature unit need to be grouped to obtain point-level input features. Local input features Mid-range input characteristics and global input features : Here, split() represents the split function.

[0035] Next, a small nonlinear mapping and activation are performed on the point-level input features to obtain the point-level features. : in, This represents a small nonlinear mapping, which can take the form of a 1×1 convolution plus activation. This represents the sigmoid function. This indicates element-wise multiplication. Point-level features are used to suppress irrelevant channel responses, highlight key details, and reduce boundary blurring.

[0036] Subsequently, depthwise separable convolution is performed on the local input features to obtain local features. : here, This represents a depthwise separable 3×3 convolution, which can also be replaced by local attention.

[0037] Next, the mid-range input features are modeled using a sparse attention mechanism, or directional convolution, to obtain the mid-range features. Mid-range features are used to represent the mid-range dependence of blood vessel orientation, which can enhance the continuous structural representation of the vascular tree. The goal is to make it easier for the model to maintain unbroken branch connections and form a more continuous vascular morphology during synthesis.

[0038] Finally, feature extraction and context aggregation are performed on the global input features to obtain the global features. Specifically, feature extraction is first performed to obtain the first intermediate global feature V, the second intermediate global feature K, and the third intermediate global feature Q: , , in, This is the first learnable global feature extraction matrix. This is a learnable second global feature extraction matrix. This is the learnable third global feature extraction matrix. Subsequently, context aggregation is performed: first, the first and second intermediate global features are aggregated with context; then, the result of this context aggregation is aggregated with the third intermediate global feature to obtain the global feature. Global features can improve the consistency of the global structure and reduce the problem of unreasonable overall reinforcement distribution.

[0039] S4: Construct multi-scale feature units through point-level features, local features, mid-range features, and global features, and use modal feature units, modal memory units, and multiple multi-scale feature units to form feature modeling units; Furthermore, the objective of this stage is to construct multi-scale feature units, thereby using modal feature units, modal memory units, and multiple multi-scale feature units to form a feature modeling unit. Specifically, in step S4, multiple multi-scale feature units are obtained, and the multi-scale feature units are connected in series. The modal feature units and modal memory units are connected in series sequentially, and the series-connected modal feature units and modal memory units are connected in parallel to the series-connected multi-scale feature units to obtain the feature modeling unit.

[0040] The specific implementation method for the above steps in this embodiment is as follows: Here, multi-scale feature units are constructed using point-level features, local features, mid-range features, and global features. The intermediate output U of the multi-scale feature unit is: Here, Concat() represents concatenation, and Fuse() represents the fuse function. The intermediate output is then added to the input features to obtain the output of the multi-scale feature unit.

[0041] Subsequently, multiple multi-scale feature units are acquired and concatenated. In this embodiment, the original image to be synthesized is first embedded as the initial input. Then, one multi-scale feature unit is used as the first multi-scale feature unit block, and the initial input is input into the first multi-scale feature unit block. Two concatenated multi-scale feature units are used as the second multi-scale feature unit block, and two concatenated multi-scale feature units are used as the third multi-scale feature unit block. The first, second, and third multi-scale feature unit blocks are then concatenated sequentially.

[0042] Two cascaded multiscale feature units form the fourth multiscale feature unit block, two cascaded multiscale feature units form the fifth multiscale feature unit block, and three cascaded multiscale feature units form the sixth multiscale feature unit block. The third multiscale feature unit block is connected before the fourth multiscale feature unit block, and the output of the last multiscale feature unit in the third multiscale feature unit block needs to undergo spatial attention before being output, and then input into the fourth multiscale feature unit block. The fourth, fifth, and sixth multiscale feature unit blocks are cascaded sequentially.

[0043] Furthermore, the modal feature units and modal memory units need to be connected in series sequentially, and then the series-connected modal feature units and modal memory units are connected in parallel to the series-connected multi-scale feature units. Specifically, in this embodiment, the series-connected modal feature units and modal memory units are connected in parallel between the third and fourth multi-scale feature unit blocks, between the fourth and fifth multi-scale feature unit blocks, and between the fifth and sixth multi-scale feature unit blocks.

[0044] The cascaded modal feature units and modal memory units are used to modulate the outputs of the third, fourth, and fifth multi-scale feature unit blocks, respectively. Specifically, during modulation, the memory representation is input into the multilayer perceptron network to obtain the scaling parameters of the j-th multi-scale feature unit block. The bias parameters of the j-th multi-scale feature unit block : , Here, the j multi-scale feature unit blocks refer to the first multi-scale feature unit block in two cascaded multi-scale feature unit blocks. Among them, This represents a scaled multilayer perceptron network for the j-th multiscale feature unit block. Let represent the biased multilayer perceptron network of the j-th multiscale feature unit block. Then, the output of the preceding multiscale feature unit block is multiplied by the scaling parameter and added to the bias parameter to complete the modulation, which is then input into the next multiscale feature unit block.

[0045] For the fourth and fifth multi-scale feature unit blocks and the fifth and sixth multi-scale feature unit blocks, after modulation, the output of the fourth multi-scale feature unit block modulated is correlated with the output of the first multi-scale feature unit block, and then input into the fifth multi-scale feature unit block. The output of the fifth multi-scale feature unit block modulated is then correlated with the output of the second multi-scale feature unit block, and then input into the sixth multi-scale feature unit block.

[0046] Finally, the output of the sixth multi-scale feature unit block is output through the output network, resulting in the generated image of the feature modeling unit, thus completing the construction of the feature modeling unit. Within each multi-scale feature unit block, the output of the previous multi-scale feature unit becomes the input feature of the next multi-scale feature unit, and the output of the last multi-scale feature unit becomes the output of the multi-scale feature unit block.

[0047] S5: Obtain the training image, input the training image into the feature modeling unit to obtain the generated image, calculate the generator total loss of the generated image, train the feature modeling unit based on the generator total loss, and use the trained feature modeling unit to obtain the target synthesized image.

[0048] Further, the objective of this stage is to calculate the total generator loss for the generated image, thereby training the feature modeling unit (IMU) to obtain the target synthesized image. Specifically, first, the training image and the corresponding enhanced target image need to be acquired. Then, the training image is input into the IMU to obtain its generated image. Next, the L1 loss between the generated and target images, as well as the least-squares adversarial loss between them, are calculated. The sum of the L1 loss and the weighted least-squares adversarial loss is the total generator loss for the generated image. With the goal of minimizing the total generator loss, the IMU is trained by adjusting the parameters of each function and unit to obtain the trained IMU. For the trained IMU, the training image is input into the modal feature unit and the first multi-scale feature unit block to obtain the enhanced target synthesized image.

[0049] The magnetic resonance image synthesis apparatus based on modal adaptation and multi-scale attention provided by the present invention will be described below. The magnetic resonance image synthesis apparatus based on modal adaptation and multi-scale attention described below can be referred to in correspondence with the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention described above.

[0050] Figure 2 A schematic diagram of the structure of a magnetic resonance image synthesis system based on modal adaptation and multi-scale attention is shown, as follows: Figure 2 As shown, the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention described above includes: Modal feature unit module 100: used to acquire target image, obtain class probability based on target image, adopt discrete sampling strategy for class probability, and complete the construction of modal feature unit; Modal memory unit module 200: used to obtain the memory matrix, and to complete the construction of the modal memory unit through the memory matrix and modal feature selection; Multi-scale feature module 300: used to construct point-level branches, local branches, mid-range branches and global branches. Point-level branches highlight details to obtain point-level features, local branches enhance local context to obtain local features, mid-range branches strengthen structural expression to obtain mid-range features, and global branches capture long-range dependencies to obtain global features. Feature modeling unit module 400: Used to construct multi-scale feature units through point-level features, local features, mid-range features and global features, and to use modal feature units, modal memory units and multiple multi-scale feature units to form feature modeling units; Target synthesized image module 500: used to acquire training images, input the training images into the feature modeling unit to obtain generated images, calculate the generator total loss of the generated images, train the feature modeling unit based on the generator total loss, and use the trained feature modeling unit to obtain the target synthesized image.

[0051] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call a computer program in the memory 830 to execute a magnetic resonance image synthesis method based on modal adaptation and multi-scale attention, the method including: S1: Obtain the target image, obtain the category probability based on the target image, and adopt a discrete sampling strategy for the category probability to complete the construction of the modal feature unit; S2: Obtain the memory matrix, and complete the construction of modal memory units through the memory matrix and modal feature selection; S3: Construct point-level branches, local branches, mid-range branches, and global branches. Point-level branches highlight details to obtain point-level features, local branches enhance local context to obtain local features, mid-range branches strengthen structural expression to obtain mid-range features, and global branches capture long-range dependencies to obtain global features. S4: Construct multi-scale feature units through point-level features, local features, mid-range features, and global features, and use modal feature units, modal memory units, and multiple multi-scale feature units to form feature modeling units; S5: Obtain the training image, input the training image into the feature modeling unit to obtain the generated image, calculate the generator total loss of the generated image, train the feature modeling unit based on the generator total loss, and use the trained feature modeling unit to obtain the target synthesized image.

[0052] Furthermore, when the computer program in the aforementioned memory 830 can be implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0053] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention provided by the above methods, the method comprising: S1: Obtain the target image, obtain the category probability based on the target image, and adopt a discrete sampling strategy for the category probability to complete the construction of the modal feature unit; S2: Obtain the memory matrix, and complete the construction of modal memory units through the memory matrix and modal feature selection; S3: Construct point-level branches, local branches, mid-range branches, and global branches. Point-level branches highlight details to obtain point-level features, local branches enhance local context to obtain local features, mid-range branches strengthen structural expression to obtain mid-range features, and global branches capture long-range dependencies to obtain global features. S4: Construct multi-scale feature units through point-level features, local features, mid-range features, and global features, and use modal feature units, modal memory units, and multiple multi-scale feature units to form feature modeling units; S5: Obtain the training image, input the training image into the feature modeling unit to obtain the generated image, calculate the generator total loss of the generated image, train the feature modeling unit based on the generator total loss, and use the trained feature modeling unit to obtain the target synthesized image.

[0054] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the magnetic resonance image synthesis methods based on modality adaptation and multi-scale attention provided above, the method comprising: S1: Obtain the target image, obtain the category probability based on the target image, and adopt a discrete sampling strategy for the category probability to complete the construction of the modal feature unit; S2: Obtain the memory matrix, and complete the construction of modal memory units through the memory matrix and modal feature selection; S3: Construct point-level branches, local branches, mid-range branches, and global branches. Point-level branches highlight details to obtain point-level features, local branches enhance local context to obtain local features, mid-range branches strengthen structural expression to obtain mid-range features, and global branches capture long-range dependencies to obtain global features. S4: Construct multi-scale feature units through point-level features, local features, mid-range features, and global features, and use modal feature units, modal memory units, and multiple multi-scale feature units to form feature modeling units; S5: Obtain the training image, input the training image into the feature modeling unit to obtain the generated image, calculate the generator total loss of the generated image, train the feature modeling unit based on the generator total loss, and use the trained feature modeling unit to obtain the target synthesized image.

[0055] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0056] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A magnetic resonance image synthesis method based on modal adaptation and multi-scale attention, characterized in that, include: S1: Obtain the target image, obtain the category probability based on the target image, and adopt a discrete sampling strategy for the category probability to complete the construction of the modal feature unit; S2: Obtain the memory matrix, and complete the construction of modal memory units through the memory matrix and modal feature selection; S3: Construct point-level branches, local branches, mid-range branches, and global branches. Point-level branches highlight details to obtain point-level features, local branches enhance local context to obtain local features, mid-range branches strengthen structural expression to obtain mid-range features, and global branches capture long-range dependencies to obtain global features. S4: Construct multi-scale feature units through point-level features, local features, mid-range features, and global features, and use modal feature units, modal memory units, and multiple multi-scale feature units to form feature modeling units; S5: Obtain the training image, input the training image into the feature modeling unit to obtain the generated image, calculate the generator total loss of the generated image, train the feature modeling unit based on the generator total loss, and use the trained feature modeling unit to obtain the target synthesized image.

2. The magnetic resonance image synthesis method based on modal adaptation and multi-scale attention according to claim 1, characterized in that, Step S1 further includes: S11: Obtain the target image, perform category probability determination on the target image, and obtain the category probability; S12: Perform discrete distribution sampling and one-hot encoding on the category probabilities to complete the construction of the modal memory unit.

3. The magnetic resonance image synthesis method based on modal adaptation and multi-scale attention according to claim 1, characterized in that, In step S2, a memory matrix is ​​obtained, and the memory matrix is ​​extracted through the modal feature selection to obtain a modal memory representation, thereby completing the construction of the modal memory unit.

4. The magnetic resonance image synthesis method based on modal adaptation and multi-scale attention according to claim 1, characterized in that, In step S3, point-level input features are obtained, and small-scale nonlinear mapping and activation are performed on the point-level input features to obtain the point-level features; Local input features are obtained, and depthwise separable convolution is performed on the local input features to obtain the local features.

5. The magnetic resonance image synthesis method based on modal adaptation and multi-scale attention according to claim 1, characterized in that, In step S3, the mid-range input features are obtained, and the mid-range input features are modeled using a sparse attention mechanism to obtain the mid-range features; Obtain global input features, perform feature extraction and context aggregation on the global input features to obtain the global features.

6. The magnetic resonance image synthesis method based on modal adaptation and multi-scale attention according to claim 1, characterized in that, In step S4, multiple multi-scale feature units are obtained, and the multi-scale feature units are connected in series. The modal feature units and modal memory units are connected in series sequentially, and the series-connected modal feature units and modal memory units are connected in parallel to the series-connected multi-scale feature units to obtain the feature modeling unit.

7. A magnetic resonance image synthesis system based on modal adaptation and multi-scale attention, used to execute the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention as described in any one of claims 1 to 6, characterized in that, include: Modal Feature Unit Module: Used to acquire target images, obtain class probabilities based on target images, and adopt a discrete sampling strategy for class probabilities to complete the construction of modal feature units; Modal memory unit module: used to obtain the memory matrix, and complete the construction of the modal memory unit by selecting modal features through the memory matrix; Multi-scale feature module: used to construct point-level branches, local branches, mid-range branches and global branches. Point-level branches highlight details to obtain point-level features, local branches enhance local context to obtain local features, mid-range branches strengthen structural expression to obtain mid-range features, and global branches capture long-range dependencies to obtain global features. Feature modeling unit module: used to construct multi-scale feature units through point-level features, local features, mid-range features and global features, and to construct feature modeling units using modal feature units, modal memory units and multiple multi-scale feature units; The target image synthesis module is used to acquire training images, input the training images into the feature modeling unit to obtain generated images, calculate the total generator loss of the generated images, train the feature modeling unit based on the total generator loss, and use the trained feature modeling unit to obtain the target image synthesis.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, characterized in that, When the program instructions are executed by the computer, the computer is able to perform the steps of the magnetic resonance image synthesis method based on modal adaptation and multi-scale attention as described in any one of claims 1 to 6.

Images