A method for generating virtual ultrasound images from magnetic resonance images based on physics-guided Transformer adversarial networks

Abstract

This invention discloses a method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network, belonging to the field of medical imaging diagnostic technology. The method includes: acquiring three-dimensional magnetic resonance images and real ultrasound images of the same anatomical location to construct a training dataset; building a dual-branch generator containing a semantic coding branch, an acoustic physical simulation branch, and a feature fusion decoder; the semantic coding branch extracts global anatomical features, and the acoustic physical simulation branch generates physical prior features; combining a dual-domain discriminator with a hybrid objective function containing adversarial and perceptual losses for end-to-end joint training; inputting the magnetic resonance image to be processed into the trained generator; and outputting a virtual ultrasound image for medical diagnosis after dual-branch collaborative processing. This method overcomes the shortcomings of traditional methods, such as insufficient physical consistency and poor feature fusion effects. The generated virtual ultrasound image possesses both anatomical accuracy and texture realism, providing reliable support for clinical diagnosis and related medical scenarios.

Description

Technical Field

[0001] This invention relates to the field of medical imaging diagnostic technology, and in particular to a method for generating virtual ultrasound images from magnetic resonance images based on a physically guided Transformer adversarial network. Background Technology

[0002] In medical imaging diagnosis, ultrasound examination is widely used in clinical practice due to its advantages of being non-invasive, real-time, and low-cost. However, image quality is easily affected by the operator's technique and the acoustic characteristics of the tissue, resulting in limited resolution and blurred texture details. Magnetic resonance imaging (MRI), on the other hand, possesses high soft tissue contrast and the ability to clearly display anatomical structures, but it is difficult to achieve real-time imaging. To overcome the inherent limitations of these two imaging technologies, image generation technology can be used to convert MRI images into virtual ultrasound images. This can provide clinicians with image references that combine anatomical accuracy and real-time performance, aiding in early disease screening, surgical planning, and multimodal image fusion diagnosis. It has significant application value, especially in scenarios where specialized ultrasound physicians are unavailable or in examinations of complex anatomical locations. This technology has become a research hotspot in the field of medical image processing.

[0003] Existing techniques for generating virtual ultrasound images from magnetic resonance imaging (MRI) images have two significant shortcomings: First, most generation methods rely on purely data-driven deep learning models, failing to fully incorporate the physical mechanisms of ultrasound imaging. This results in generated images that, while visually similar to real ultrasound images, lack the support of acoustic propagation laws, leading to deviations from real ultrasound images in terms of physical characteristics such as tissue interface reflection and depth attenuation. Consequently, they fail to meet the requirements of clinical diagnosis for the physical realism of images. Second, the feature extraction and fusion strategies of existing methods are not perfect. They either focus only on the anatomical semantic features of MRI images while ignoring the unique texture and attenuation features of ultrasound, or the feature fusion process lacks targeted design. This makes it difficult for the generated virtual ultrasound images to simultaneously achieve anatomical structure correspondence and texture detail reproduction, failing to accurately reproduce the clinical characteristics of real ultrasound images and limiting their widespread application in actual clinical scenarios. Summary of the Invention

[0004] To overcome the shortcomings and deficiencies of existing technologies, this invention provides a method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network.

[0005] The technical solution adopted in this invention is a method for generating virtual ultrasound images from magnetic resonance images based on a physically guided Transformer adversarial network, comprising the following steps: S1. Collect three-dimensional magnetic resonance images of the same anatomical site as source domain data and real ultrasound images as target domain data to construct a training dataset; S2, a dual-branch generator network is constructed, which includes a semantic coding branch, an acoustic physics simulation branch, and a feature fusion decoder. The semantic coding branch uses a global feature extraction module based on the Transformer architecture to extract global anatomical features. The acoustic physics simulation branch maps physical parameter maps through a lightweight convolutional network and generates physical prior feature maps through a differentiable ray tracking layer. The feature fusion decoder concatenates the features of the two branches to restore the image resolution. S3, construct a dual-domain discriminator network containing a spatial domain discriminator and a frequency domain discriminator; S4 defines a hybrid objective function that includes adversarial loss, physical consistency loss, and perceptual loss, and performs end-to-end joint training on the generator and discriminator. S5 inputs the magnetic resonance image to be processed into the trained generator, and outputs a virtual ultrasound image for medical diagnosis through dual-branch collaborative processing.

[0006] Furthermore, the reflection coefficient in the acoustic physics simulation branch is calculated using the following formula: ,in, Image coordinates The reflection coefficient at that location, coordinates The corresponding pseudo-acoustic impedance value, coordinates The corresponding pseudo-acoustic impedance value, A small adjustment constant is used to avoid the denominator approaching zero.

[0007] Furthermore, the cumulative attenuation in the acoustic physics simulation branch is calculated using the following formula: ,in, For depth The cumulative attenuation coefficient at that location, coordinates The corresponding tissue attenuation coefficient value, This is the attenuation correction factor. From initial depth to target depth The cumulative index variable.

[0008] Furthermore, the physical prior feature map is generated using the following formula: ,in, coordinates The physical prior eigenvalues ​​at that location, This is the absolute value of the reflection coefficient. This is the cumulative attenuation coefficient. This is a characteristic intensity adjustment factor.

[0009] Furthermore, the physical consistency loss is calculated using the following formula: ,in, This represents the physical consistency loss value. The virtual ultrasound image output by the generator. These are the weighting coefficients for physical prior features. This is a physical prior feature map. This is the grayscale offset. For L1 norm operations, This is the loss balance coefficient.

[0010] Further, S1 includes the following sub-steps: S11, screening three-dimensional magnetic resonance images and real ultrasound images of the same anatomical site to ensure that the images include different tissue types and anatomical structures of the site; S12, performing targeted denoising processing on the screened magnetic resonance images and ultrasound images to remove interference signals unrelated to the anatomical structure; S13, performing geometric correction on the denoised images to adjust the image spatial orientation to conform to a preset imaging benchmark; S14, dividing the corrected images into training and validation sets according to a ratio, maintaining the consistency of the anatomical structure distribution among different datasets during the division process to form a complete training dataset.

[0011] Further, S2 includes the following sub-steps: S21, constructing a semantic encoding branch, first dividing the input magnetic resonance image into fixed-size patches and mapping them to linear embedding vectors, and then calculating the correlation information between patches to extract global anatomical structural features through a stacked multi-layer moving window self-attention mechanism Swin Transformer module; S22, constructing a parameter prediction subnetwork of the acoustic physics simulation branch, using a two-layer convolutional network to transform the shallow features of the magnetic resonance image, and outputting a pseudo acoustic impedance map and an attenuation coefficient map; S23, embedding a differentiable ray tracing layer in the acoustic physics simulation branch, simulating sound wave propagation along a preset scan line direction, calculating the reflection intensity and depth cumulative attenuation at different interfaces, the calculation process of the ray tracing layer consists of differentiable tensor operations, supporting gradient backpropagation. In the process of ray tracking, in order to ensure the back propagation of gradients, bilinear interpolation or trilinear interpolation mechanism is used to sample the physical parameters of non-integer coordinate points; S24, a feature fusion decoder is built, which splices semantic features and physical prior features through the channel dimension, and gradually improves the feature map resolution through multiple upsampling convolutional layers to output a virtual ultrasound image.

[0012] Further, step S3 includes the following sub-steps: S31, constructing a spatial domain discriminator by building a multi-layer convolutional network using a PatchGAN structure and setting a sliding window to divide the input image into local blocks; S32, the spatial domain discriminator extracts the texture features and grayscale distribution features of each local block through convolution operations and outputs the authenticity judgment result of each local block; S33, constructing a frequency domain discriminator by embedding a fast Fourier transform module to convert the input real ultrasound image and virtual ultrasound image to the frequency domain space; S34, the frequency domain discriminator separates the amplitude spectrum and phase spectrum of the frequency domain image, uses them as dual-channel input to a lightweight convolutional network, and outputs the authenticity judgment result of the frequency domain features through convolution operations.

[0013] Further, S4 includes the following sub-steps: S41, setting the weight coefficients of the mixed objective function and clarifying the proportions of adversarial loss, physical consistency loss, and perceptual loss in the total loss; S42, calculating the adversarial loss term using least squares generative adversarial loss and fusing the outputs of the spatial domain discriminator and the frequency domain discriminator; S43, extracting deep features of the generated image and the real ultrasound image using a pre-trained deep network, and calculating the L1 distance between the two as the perceptual loss term; S44, combining the loss terms according to the set weight coefficients to form the total loss function, and adopting an alternating training strategy, first fixing the generator and updating the discriminator parameters, then fixing the discriminator and updating the generator parameters, and iteratively optimizing the network.

[0014] Further, S5 includes the following sub-steps: S51, selecting the trained generator network, removing the auxiliary training modules used during training, and retaining the semantic coding branch, acoustic physical simulation branch, and feature fusion decoder; S52, receiving the three-dimensional magnetic resonance image to be processed and inputting it into the semantic coding branch and acoustic physical simulation branch of the generator; S53, the semantic coding branch and acoustic physical simulation branch simultaneously process the input image and output the global anatomical feature map and physical prior feature map, respectively; S54, the feature fusion decoder receives the feature maps output by the two branches, and after splicing, upsampling, and convolution operations, outputs a virtual ultrasound image corresponding to the input magnetic resonance image.

[0015] Beneficial Effects: This invention proposes a method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network. It introduces an acoustic physics simulation branch, generating physical prior features by simulating core physical phenomena in the sound wave propagation process. This ensures that the generated virtual ultrasound images strictly adhere to the laws of acoustic propagation, maintaining a high degree of consistency with real ultrasound images in terms of tissue interface reflection and depth attenuation characteristics, thus completely solving the problem of insufficient physical realism in purely data-driven methods. Simultaneously, the semantic encoding branch accurately extracts global anatomical features from the magnetic resonance image using a moving window self-attention mechanism. The feature fusion decoder achieves deep fusion of the two types of features through targeted design, ensuring both accurate correspondence of anatomical structures and perfect restoration of the unique texture details of ultrasound images, effectively balancing anatomical accuracy and texture realism. Furthermore, a dual-domain discriminator constrains the generated image from both spatial and frequency domains, and the hybrid objective function further improves the generation quality through multi-loss collaborative optimization. This enables the output virtual ultrasound image to simultaneously meet the requirements of clinical diagnosis for physical realism, structural accuracy, and detail restoration, providing reliable support for multimodal image fusion diagnosis, surgical planning, and other scenarios, significantly expanding the clinical application scope of this technology. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention. Figure 2 This is a flowchart of the method step S1 of the present invention. Figure 3 This is a flowchart of step S3 of the method of the present invention. Figure 4 This is a flowchart of method step S3 of the present invention; Figure 5 This is a flowchart of method step S4 of the present invention; Figure 6 This is a flowchart of step S5 of the method of the present invention. Detailed Implementation

[0018] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0019] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0020] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0021] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0022] like Figure 1 As shown, a method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network includes the following steps: S1. Collect three-dimensional magnetic resonance images of the same anatomical site as source domain data and real ultrasound images as target domain data to construct a training dataset; Specifically, step S1 involves constructing the training dataset. In practice, three-dimensional magnetic resonance imaging (MRI) images and real ultrasound images of the same anatomical location are first acquired. The MRI image acquisition resolution is set to 1×1×1.5, with a single image pixel size of 512×512×128. The ultrasound images are acquired using a linear array probe with a center frequency of 7.5MHz, an imaging depth of 15-20, and a pixel size of 512×512. Both types of images include different tissue types such as normal tissue and diseased tissue, as well as anatomical structures of subjects of different ages and body types, ensuring data diversity. After acquisition, the images undergo targeted denoising processing. The MRI images are denoised using a non-local means denoising algorithm with a neighborhood window size of 7×7 and a search window size of 21×21. The ultrasound images are denoised using a wavelet thresholding algorithm with a decomposition level of 3 and a threshold coefficient of 0.8, thoroughly removing interference signals unrelated to anatomical structures, such as equipment noise and artifacts. Geometric correction was then performed. Using the magnetic resonance imaging (MRI) images as a reference, the spatial orientation of the ultrasound images was adjusted using a rigid registration algorithm. The registration iteration count was set to 200 times, and the convergence threshold was set to 1e-6 to ensure the spatial correspondence accuracy of the anatomical structures in the two image types. Finally, the dataset was divided into training and validation sets in a 7:3 ratio. Stratified sampling was used during the partitioning process to maintain the consistency of the distribution of different tissue types and lesion types in the training and validation sets. This resulted in a training dataset containing at least 1000 paired images, providing reliable data support for subsequent network training.

[0023] S2, a dual-branch generator network is constructed, which includes a semantic coding branch, an acoustic physics simulation branch, and a feature fusion decoder. The semantic coding branch uses a global feature extraction module based on the Transformer architecture to extract global anatomical features. The acoustic physics simulation branch maps physical parameter maps through a lightweight convolutional network and generates physical prior feature maps through a differentiable ray tracking layer. The feature fusion decoder concatenates the features of the two branches to restore the image resolution. Specifically, step S2 involves building a dual-branch generator network. The semantic encoding branch is constructed using the Swin Transformer module. First, the input 3D magnetic resonance image is divided into fixed-size patches of 16×16×16. Each patch is mapped to an embedding vector of dimension 512 through linear projection. Then, four layers of Swin Transformer modules are stacked. The moving window size of each module is set to 8×8×8, the number of self-attention heads is 8, and the dimension of the feedforward network hidden layer is 2048. Through in-window self-attention calculation and cross-window feature interaction, the global anatomical structure features of the image are accurately extracted, and the feature map output dimension is 64×64×32×512. The acoustic physics simulation branch consists of a parameter prediction subnetwork and a differentiable ray tracking layer. The parameter prediction subnetwork uses a two-layer convolutional network. The first layer has a kernel size of 3×3×3, 64 kernels, a stride of 1, and a padding of 1. The second layer has a kernel size of 3×3×3, 2 kernels, a stride of 1, and a padding of 1. It transforms the shallow features of the magnetic resonance image and outputs a pseudo-acoustic impedance map and an attenuation coefficient map. The differentiable ray tracking layer sets 1024 uniformly distributed scan lines. The preset scan line direction can be parallel scanning (corresponding to linear array probes) or fan-shaped scanning (corresponding to convex array or phased array probes). The scanning angle range is -45° to 45°, and the sampling interval is 0.1. It simulates the sound wave propagation process along the scan line direction, calculates the reflection intensity and depth cumulative attenuation of different tissue interfaces, and generates a physical prior feature map with dimensions of 64×64×32×512. The feature fusion decoder adopts a U-Net-like structure, which concatenates the feature maps output by the dual branches through the channel dimension to form a fusion feature with a dimension of 64×64×32×1024. Then, the resolution is gradually improved through 4 upsampling convolutional layers. Each upsampling layer uses transposed convolution with a kernel size of 2×2×2 and a stride of 2. Finally, the output is a virtual ultrasound image with the same size as the input magnetic resonance image, realizing a complete mapping from feature extraction to image generation.

[0024] S3, construct a dual-domain discriminator network containing a spatial domain discriminator and a frequency domain discriminator; Specifically, step S3 involves constructing a dual-domain discriminator network. The spatial domain discriminator adopts a PatchGAN structure, consisting of 5 convolutional layers. The first layer has a kernel size of 4×4×4, a number of 64 kernels, a stride of 2, and padding of 1. Subsequent layers maintain a kernel size of 4×4×4, with the number of kernels doubling sequentially to 128, 256, 512, and 1, respectively, with a stride of 2 and padding of 1. The last layer uses a sigmoid activation function and sets a 16×16×8 sliding window to divide the input image into local blocks. Each window corresponds to a realism judgment result. Through convolution operations, the texture features and grayscale distribution features of each local block are extracted layer by layer, and a judgment matrix with a dimension of 32×32×8×1 is output to achieve accurate discrimination of the realism of local details in the image. The frequency domain discriminator embeds a Fast Fourier Transform (FFT) module. It first transforms the input real and virtual ultrasound images into the frequency domain space to obtain the amplitude and phase spectra of the frequency domain images. The amplitude spectrum is then subjected to a logarithmic transform to compress the dynamic range. Subsequently, the amplitude and phase spectra are used as dual-channel inputs to a lightweight convolutional network. This network consists of three convolutional layers with kernel sizes of 3×3×3 and numbers of 32, 64, and 1, respectively, a stride of 1, and padding of 1. The intermediate layers use the ReLU activation function, and the output layer uses the sigmoid activation function. Through convolution operations, the amplitude distribution pattern and phase consistency features of the frequency domain features are extracted. The output is a frequency domain authenticity judgment result with a dimension of 64×64×16×1, which supplements the constraints from the frequency domain dimension to ensure that the frequency distribution of the generated image is consistent with the real ultrasound image.

[0025] S4 defines a hybrid objective function that includes adversarial loss, physical consistency loss, and perceptual loss, and performs end-to-end joint training on the generator and discriminator. Specifically, step S4 involves defining the hybrid objective function and jointly training the network. First, the weight coefficients of the hybrid objective function are set, with the adversarial loss weight coefficient set to 1.0, the physical consistency loss weight coefficient set to 0.8, and the perceptual loss weight coefficient set to 0.5, clarifying the proportion of each loss term in the total loss. The adversarial loss uses least-squares generative adversarial loss, calculating the output differences between the spatial domain discriminator and the frequency domain discriminator for the generated and real images respectively. The loss results of the two discriminators are then fused in a 1:1 ratio to form the final adversarial loss term, effectively mitigating the mode collapse problem in traditional GAN ​​training. The perceptual loss uses a pre-trained VGG19 network as a feature extractor, selecting the output features of the 1st, 3rd, 5th, and 7th convolutional layers. The L1 distance between the generated image and the real ultrasound image at each layer is calculated, and the distances at each layer are weighted and summed with weights of 0.1, 0.2, 0.3, and 0.4 to form the perceptual loss term, ensuring that the high-level semantic features of the generated image are consistent with the real image. The physical consistency loss is calculated by altering the L1 distance between the generated image and the physical prior feature map after linear transformation, constraining the generated image to conform to the physical laws of ultrasound imaging. The total loss function is the weighted sum of each loss term according to a set weight coefficient. The training process uses the Adam optimizer, with the learning rate of both the generator and discriminator set to 2e-4, the weight decay coefficient set to 1e-5, and the batch size to 4. An alternating training strategy is adopted, with the generator updating the discriminator parameters once per iteration, and the learning rate adjusted once every 50 iterations. The learning rate decay factor is 0.9. The training is stopped after 20,000 iterations to ensure that the network converges to the optimal state.

[0026] S5 inputs the magnetic resonance image to be processed into the trained generator, and outputs a virtual ultrasound image for medical diagnosis through dual-branch collaborative processing.

[0027] Specifically, step S5 involves virtual ultrasound image generation. First, the trained generator network is selected, and auxiliary training modules used for gradient calculation and loss monitoring during training are removed, retaining only the semantic encoding branch, acoustic physics simulation branch, and feature fusion decoder to reduce redundant computation and improve inference speed. Then, the 3D magnetic resonance image to be processed is received. This image must have the same acquisition parameters as the training dataset, a resolution of at least 1×1×2, a pixel size of at least 256×256×64, and be in DICOM format. It can be directly input into the generator without additional preprocessing. The generator's semantic encoding branch and acoustic physics simulation branch process the input image synchronously. The semantic encoding branch quickly extracts the global anatomical structure features of the image according to preset tile division and embedding rules, with a processing time of no more than 0.5 seconds. The acoustic physics simulation branch simultaneously predicts the physical parameter map and generates a physical prior feature map through a differentiable ray tracking layer, ensuring the temporal consistency of the feature outputs from both branches. The feature fusion decoder receives the feature maps output from the dual branches. It first integrates the two types of features through channel splicing, and then gradually restores the image resolution through multiple upsampling convolutional layers. During the upsampling process, a combination of interpolation and convolution is used to avoid loss of feature information. Finally, the number of output channels is adjusted to 1 through a 1×1×1 convolutional kernel to obtain a single-channel virtual ultrasound image. The pixel size of the output image is consistent with that of the input magnetic resonance image, and the grayscale range is 0-255, which conforms to the observation habits of clinical ultrasound images and can be directly used for clinical diagnostic reference or subsequent medical applications.

[0028] Preferably, the reflection coefficient in the acoustic physics simulation branch is calculated using the following formula: ,in, Image coordinates The reflection coefficient at that location, coordinates The corresponding pseudo-acoustic impedance value, coordinates The corresponding pseudo-acoustic impedance value, A small adjustment constant is used to avoid the denominator approaching zero.

[0029] Specifically, the calculation method for the reflection coefficient in the acoustic physics simulation branch is a core step in generating physical prior features, directly affecting the realism of tissue interface reflection effects in virtual ultrasound images. In practice, the reflection coefficient calculation is based on the pseudo-acoustic impedance map output by the acoustic physics simulation branch. Pairwise operations are performed on each coordinate point in the image and its adjacent coordinate points. The adjacent coordinate point is selected as the next coordinate point in the depth direction from the current coordinate point, ensuring the reflection law when the combined sound wave is perpendicularly incident. A small adjustment constant is introduced during the calculation process, set to 0.001. This value has been verified through multiple experiments, effectively avoiding calculation anomalies caused by the denominator approaching zero, and without significantly interfering with the true value of the reflection coefficient. In the specific calculation, the pseudo-acoustic impedance value corresponding to the current coordinate point and the pseudo-acoustic impedance value corresponding to the next coordinate point in the depth direction are first obtained. The difference between the two is calculated and added to the small adjustment constant to obtain the numerator. The sum of the two is then added to twice the small adjustment constant to obtain the denominator. The reflection coefficient of that coordinate point is obtained by the ratio of the numerator to the denominator. The reflection coefficient is strictly controlled within the range of -1 to 1. A positive value indicates that the reflected wave is in phase with the incident wave, while a negative value indicates that they are out of phase. The absolute value reflects the reflection intensity. The reflection coefficient calculated by this method can accurately characterize the acoustic reflection properties of different tissue interfaces, ensuring that the subsequently generated physical prior feature map exhibits grayscale changes at tissue interfaces that conform to the laws of real ultrasound imaging. This provides crucial support for improving the physical consistency of virtual ultrasound images, ensuring that the boundaries of different tissues in the generated image are clear and the reflection intensity is reasonable.

[0030] Preferably, the cumulative attenuation in the acoustic physics simulation branch is calculated using the following formula: ,in, For depth The cumulative attenuation coefficient at that location, coordinates The corresponding tissue attenuation coefficient value, This is the attenuation correction factor. From initial depth to target depth The cumulative index variable.

[0031] Specifically, the method for calculating cumulative attenuation in the acoustic physics simulation branch is used to simulate the energy attenuation effect of sound waves propagating in tissues. This method is a crucial technique for ensuring the rationality of grayscale gradation in the depth direction of virtual ultrasound images. In practice, the cumulative attenuation is calculated along the depth direction, starting from the initial depth of the image and accumulating point by point to the target depth. The cumulative index variable increments sequentially from 0 to the index value corresponding to the target depth, ensuring coverage of the complete sound wave propagation path. The tissue attenuation coefficient value is output by the parameter prediction subnet of the acoustic physics simulation branch. Its value is directly related to the tissue type; for example, the attenuation coefficient value of bone tissue is significantly higher than that of soft tissue, and the attenuation coefficient value of adipose tissue is lower than that of muscle tissue. The attenuation correction coefficient is set to 0.01. This parameter is used to adjust the overall intensity of the attenuation, making it conform to the attenuation patterns of tissues at different depths in clinical ultrasound imaging. This avoids excessive attenuation leading to the complete obscuration of deep tissue images, or insufficient attenuation resulting in indistinct grayscale differences between shallow and deep tissues. In the calculation process, firstly, for each depth coordinate point, the corresponding tissue attenuation coefficient value is obtained and multiplied by the attenuation correction coefficient to obtain the instantaneous attenuation amount at that point. Then, starting from the initial depth, the instantaneous attenuation amounts of each coordinate point are accumulated sequentially to obtain the total attenuation amount from the initial depth to the target depth. Finally, the total attenuation amount is converted using an exponential function to obtain the cumulative attenuation coefficient at the target depth. The value of this cumulative attenuation coefficient ranges from 0 to 1, gradually decreasing with increasing depth. This accurately simulates the physical phenomenon of energy attenuation with increasing distance during sound wave propagation, resulting in a natural grayscale reduction effect in the depth direction of the generated virtual ultrasound image, enhancing the realism of the image and its clinical reference value.

[0032] Preferably, the physical prior feature map is generated using the following formula: ,in, coordinates The physical prior eigenvalues ​​at that location, This is the absolute value of the reflection coefficient. This is the cumulative attenuation coefficient. This is a characteristic intensity adjustment factor.

[0033] Specifically, the generation of the physical prior feature map constructs a physical prior feature that combines reflection and attenuation characteristics by fusing the reflection coefficient and cumulative attenuation. This provides a foundational feature that conforms to the physical laws of ultrasound imaging for subsequent feature fusion, and is a crucial link connecting acoustic physical simulation and feature fusion. In practice, the feature intensity adjustment factor is set to 0.8 to 1.5. This value was determined through extensive comparative experiments and can moderately enhance the intensity of the physical prior feature without changing the relative relationships of the features. This allows the physical prior feature to play its full role in the subsequent feature fusion process, while avoiding fusion imbalance caused by excessively high or low feature intensity. The generation process performs point-by-point calculations on a per-coordinate basis in the image. First, the calculated reflection coefficient is obtained, and its absolute value is taken to eliminate the influence of phase differences on the feature intensity, retaining only the reflection intensity information to ensure the non-negativity and comparability of the feature values. Then, the calculated cumulative attenuation coefficient is obtained, which reflects the degree of energy attenuation when the sound wave propagates to the current coordinate point. Multiplying the absolute value of the reflection coefficient by the cumulative attenuation coefficient yields the fundamental feature value that fuses reflection and attenuation characteristics. This fundamental feature value is then multiplied by a feature intensity adjustment factor to obtain the final physical prior feature value. The physical prior feature values ​​at each coordinate point collectively constitute the physical prior feature map. The size of this feature map is consistent with the anatomical feature map output by the semantic coding branch, ensuring the feasibility of subsequent channel dimension stitching. The physical prior feature map generated by this method accurately integrates the core physical characteristics of ultrasound imaging, guiding the generator to fully incorporate acoustic propagation laws during image generation. This results in a final virtual ultrasound image that possesses both clear anatomical structures and exhibits physical characteristics consistent with real ultrasound images, effectively improving the clinical applicability of the image.

[0034] Preferably, the physical consistency loss is calculated using the following formula: ,in, This represents the physical consistency loss value. The virtual ultrasound image output by the generator. These are the weighting coefficients for physical prior features. This is a physical prior feature map. This is the grayscale offset. For L1 norm operations, This is the loss balance coefficient.

[0035] Specifically, the calculation of physical consistency loss is crucial for ensuring the generated image conforms to the physical laws of ultrasound imaging. By quantifying the difference between the generated image and the physical prior feature map, it guides the network's iterative optimization and directly determines the level of physical realism of the generated image. In practice, the loss balance coefficient is set to 0.8. This weighting coefficient matches the weighting coefficient of the physical consistency loss in the hybrid objective function, ensuring that the physical consistency constraint occupies a reasonable proportion in the overall optimization process. This effectively improves the physical realism of the generated image without excessively suppressing the constraint effect of other loss terms. The grayscale offset is set to 10. This parameter is used to compensate for the difference in grayscale baseline between the physical prior feature map and the generated image. Since the numerical range of the physical prior feature map is concentrated between 0 and 1.2, while the grayscale range of the generated image is 0 to 255, adjusting the grayscale offset can bring the grayscale distribution of the two into a similar range, improving the effectiveness of loss calculation. The physical prior feature weighting coefficient is set to 200. This coefficient is used to scale the numerical range of the physical prior feature map to a level similar to that of the generated image, avoiding imbalance in loss calculation due to differences in numerical magnitude. In the calculation process, the physical prior feature map is first multiplied by the physical prior feature weight coefficient, and then added to the grayscale offset to complete the grayscale calibration of the physical prior feature map, ensuring that its numerical range is consistent with the virtual ultrasound image output by the generator. Subsequently, the L1 distance between the calibrated physical prior feature map and the virtual ultrasound image is calculated. This distance accurately quantifies the grayscale difference between the two at each pixel. This L1 distance is then multiplied by the loss balance coefficient to obtain the final physical consistency loss value. During network training, this loss value updates the generator's network parameters through backpropagation, continuously reducing the difference between the generated image and the physical prior feature map. This ensures that the generated image maintains consistency with the real ultrasound image in key physical properties such as reflectivity and attenuation characteristics, fundamentally solving the problem of insufficient physical realism in purely data-driven methods.

[0036] Preferred, such as Figure 2 As shown, step S1 includes the following sub-steps: S11, screening three-dimensional magnetic resonance images and real ultrasound images of the same anatomical site to ensure that the images include different tissue types and anatomical structures of the site; S12, performing targeted denoising processing on the screened magnetic resonance images and ultrasound images to remove interference signals unrelated to the anatomical structure; S13, performing geometric correction on the denoised images to adjust the image spatial orientation to conform to the preset imaging benchmark; S14, dividing the corrected images into training and validation sets according to the proportion, maintaining the consistency of the anatomical structure distribution among different datasets during the division process to form a complete training dataset.

[0037] Specifically, step S1 consists of four sub-steps: S11, when screening 3D MRI images and real ultrasound images of the same anatomical location, ensures that the images include different tissue types such as normal tissue, diseased tissue, adipose tissue, and muscle tissue, as well as different anatomical structures such as organ outlines, blood vessel distribution, and glandular structures, to ensure that the data covers common clinical anatomical scenarios; S12, performs targeted denoising on the screened images, using a non-local mean denoising algorithm for MRI images with a neighborhood window size of 7×7 and a search window size of 21×21, and using a wavelet threshold denoising algorithm for ultrasound images with a decomposition layer of 3 and a threshold coefficient of 0.8. Through precise setting of algorithm parameters, interference signals unrelated to anatomical structures, such as electronic noise from the equipment and scanning artifacts, are efficiently removed; S13, geometric... During calibration, the magnetic resonance image was used as the reference template, and a rigid registration algorithm was adopted with 200 iterations and a convergence threshold of 1e-6. By adjusting the translation and rotation parameters of the ultrasound image, its spatial orientation was made completely consistent with the imaging reference of the magnetic resonance image, ensuring the spatial correspondence accuracy of the anatomical structures of the two types of images. In S14, when dividing the training set and the validation set, a stratified sampling method was used at a ratio of 7:3 to maintain the consistent distribution ratio of various tissue types and lesion types among different datasets. Finally, a training dataset including at least 1000 pairs of images was formed. Each step was progressive, from data screening, purification, calibration to division, to comprehensively ensure the diversity, purity and consistency of the training data, providing high-quality data support for subsequent network training and ensuring that the trained network model has good generalization ability.

[0038] Preferred, such as Figure 3 As shown, S2 includes the following sub-steps: S21, constructing a semantic encoding branch, first dividing the input magnetic resonance image into fixed-size patches and mapping them to linear embedding vectors, then using a multi-layer moving window self-attention mechanism Swin Transformer module to solve the 3D computing resource bottleneck, adopting a block training strategy instead of full-image input to reduce memory usage and improve the generation quality of local details, calculating the correlation information between patches to extract global anatomical structure features; S22, constructing a parameter prediction subnetwork for the acoustic physics simulation branch, using a two-layer convolutional network to transform the shallow features of the magnetic resonance image, outputting a pseudo acoustic impedance map and attenuation coefficient map; S23, embedding a differentiable ray tracking layer in the acoustic physics simulation branch, simulating sound wave propagation along a preset scan line direction, calculating the cumulative attenuation of reflection intensity and depth at different interfaces; S24, constructing a feature fusion decoder, splicing semantic features and physical prior features through the channel dimension, gradually improving the feature map resolution through multiple upsampling convolutional layers, and outputting a virtual ultrasound image.

[0039] Specifically, in step S2, the four sub-steps are implemented. In S21, when building the semantic encoding branch, the input magnetic resonance image is first divided into fixed-size patches of 16×16×16. Each patch is mapped to an embedding vector of dimension 512 through linear projection. Then, four layers of moving window self-attention mechanism Swin are stacked. The Transformer module, with each layer having a moving window size of 8×8×8, 8 self-attention heads, and a feedforward network hidden layer dimension of 2048, accurately extracts global anatomical features of the image through in-window self-attention calculation and cross-window feature interaction. S22 constructs the parameter prediction subnetwork for the acoustic physics simulation branch, employing a two-layer convolutional network. The first layer has a kernel size of 3×3×3, 64 kernels, a stride of 1, and padding of 1; the second layer has a kernel size of 3×3×3, 2 kernels, a stride of 1, and padding of 1. Through convolution operations, it transforms shallow features of the magnetic resonance image, accurately outputting pseudo-acoustic impedance maps and attenuation coefficient maps. S23 embeds a differentiable ray tracking layer in the acoustic physics simulation branch, setting 1024 uniformly distributed scan lines. The system uses line drawing and scanning angles ranging from -45° to 45° with a sampling interval of 0.1 to simulate the propagation of sound waves along a preset scanning line, accurately calculating the reflection intensity and depth cumulative attenuation at different tissue interfaces. The S24 module builds a feature fusion decoder, which splices semantic features and physical prior features along the channel dimension to form a fusion feature with doubled dimensions. This feature map resolution is then progressively improved through four upsampling convolutional layers. Each upsampling layer uses transposed convolution with a kernel size of 2×2×2 and a stride of 2. The final output is a virtual ultrasound image with the same size as the input MRI image. Each step works collaboratively to achieve a complete process from anatomical feature extraction, physical parameter prediction, physical process simulation to feature fusion generation, ensuring that the generated image possesses both anatomical accuracy and physical realism.

[0040] Preferred, such as Figure 4 As shown, step S3 includes the following sub-steps: S31, constructing a spatial domain discriminator by building a multi-layer convolutional network using a PatchGAN structure and setting a sliding window to divide the input image into local blocks; S32, the spatial domain discriminator extracts the texture features and grayscale distribution features of each local block through convolution operations and outputs the authenticity judgment result of each local block; S33, constructing a frequency domain discriminator by embedding a fast Fourier transform module to convert the input real ultrasound image and virtual ultrasound image to the frequency domain space; S34, the frequency domain discriminator separates the amplitude spectrum and phase spectrum of the frequency domain image, uses them as dual-channel input to a lightweight convolutional network, and outputs the authenticity judgment result of the frequency domain features through convolution operations.

[0041] Specifically, step S3 is implemented in four sub-steps. In S31, when constructing the spatial domain discriminator, a 5-layer convolutional network is built using the PatchGAN structure. The first layer has a kernel size of 4×4×4, a number of kernels of 64, a stride of 2, and padding of 1. Subsequent layers maintain a kernel size of 4×4×4, with the number of kernels doubling sequentially to 128, 256, 512, and 1, with a stride of 2 and padding of 1. The last layer uses the sigmoid activation function and sets a 16×16×8 sliding window to divide the input image into local blocks, ensuring accurate discrimination of local image details. In S32, the spatial domain discriminator extracts the texture features and gray-level distribution features of each local block through layer-by-layer convolution operations. Texture features include speckle noise texture and tissue interface texture unique to ultrasound images. Gray-level distribution features reflect statistical information such as the mean and variance of gray levels in different regions. Finally, the system outputs the authenticity judgment result for each local block. When building the frequency domain discriminator in S33, a Fast Fourier Transform (FFT) module is embedded to quickly convert the input real and virtual ultrasound images to the frequency domain space, obtaining the amplitude and phase spectra of the frequency domain images, laying the foundation for frequency domain feature extraction. The frequency domain discriminator in S34 first separates the amplitude and phase spectra of the frequency domain images and uses them as dual-channel input to a 3-layer lightweight convolutional network. The convolution kernel size is 3×3×3, with 32, 64, and 1 kernels respectively, a stride of 1, and padding of 1. The intermediate layers use the ReLU activation function, and the output layer uses the sigmoid activation function. The amplitude distribution pattern and phase consistency features of the frequency domain features are extracted through convolution operations, and the frequency domain authenticity judgment result is output. Each step constructs a discrimination mechanism from both spatial and frequency domain dimensions, comprehensively constraining the local details, texture features, grayscale distribution, and frequency characteristics of the generated image, ensuring that the generated image is highly similar to the real ultrasound image.

[0042] Preferred, such as Figure 5 As shown, step S4 includes the following sub-steps: S41, setting the weight coefficients of the mixed objective function and clarifying the proportions of adversarial loss, physical consistency loss, and perceptual loss in the total loss; S42, calculating the adversarial loss term using least squares generation adversarial loss and fusing the outputs of the spatial domain discriminator and the frequency domain discriminator; S43, extracting deep features of the generated image and the real ultrasound image using a pre-trained deep network, and calculating the L1 distance between them as the perceptual loss term; S44, combining the loss terms according to the set weight coefficients to form the total loss function, and adopting an alternating training strategy, first fixing the generator and updating the discriminator parameters, then fixing the discriminator and updating the generator parameters, and iteratively optimizing the network.

[0043] Specifically, step S4 is implemented in four sub-steps. In S41, when setting the weight coefficients of the mixed objective function, the weight coefficients of the adversarial loss are explicitly set to 1.0, the weight coefficient of the physical consistency loss is 0.8, and the weight coefficient of the perceptual loss is 0.5. Through scientific weight allocation, the constraints of each loss term are balanced. In S42, when calculating the adversarial loss term, the least squares generative adversarial loss is used. The output differences between the spatial domain discriminator and the frequency domain discriminator for the generated image and the real image are calculated separately. Then, the loss results of the two types of discriminators are fused in a 1:1 ratio, which effectively alleviates the pattern collapse problem in the training of traditional generative adversarial networks. To improve training stability, in S43, when calculating the perceptual loss term, a pre-trained VGG19 network is used as a feature extractor. The output features of the 1st, 3rd, 5th, and 7th convolutional layers of the network are selected. The L1 distance between the generated image and the real ultrasound image on each layer feature is calculated, and then weighted and summed with weights of 0.1, 0.2, 0.3, and 0.4 to ensure that the high-level semantic features of the generated image are consistent with the real image. In S44, when combining the total loss function, each loss term is weighted and summed according to the set weight coefficients. The Adam optimizer is used during the training process, and the learning rate of both the generator and the discriminator is set to 2e-4, with a weight decay coefficient of 1e. -5 The batch size is 4, and an alternating training strategy is adopted. The generator updates the discriminator parameters once every iteration, and the learning rate is adjusted once every 50 iterations. The learning rate decay factor is 0.9. The training stops after 20,000 iterations. Each step, from loss function design and loss term calculation to training strategy formulation, forms a complete network optimization system to ensure that the network converges to the optimal state quickly and improves the quality of generated images.

[0044] Preferred, such as Figure 6 As shown, S5 includes the following sub-steps: S51, select the trained generator network, remove the auxiliary training modules used during training, and retain the semantic coding branch, acoustic physical simulation branch, and feature fusion decoder; S52, receive the three-dimensional magnetic resonance image to be processed and input it into the semantic coding branch and acoustic physical simulation branch of the generator; S53, the semantic coding branch and acoustic physical simulation branch process the input image synchronously and output the global anatomical feature map and physical prior feature map respectively; S54, the feature fusion decoder receives the feature maps output by the two branches, and after splicing, upsampling, and convolution operations, outputs a virtual ultrasound image corresponding to the input magnetic resonance image.

[0045] Specifically, step S5 is implemented in four sub-steps: S51 When selecting the trained generator network, the auxiliary training modules used for gradient calculation, loss monitoring, and parameter updates during training are removed, and only the semantic coding branch, acoustic physical simulation branch, and feature fusion decoder are retained to reduce redundant calculation steps and improve inference speed; S52 When receiving the 3D magnetic resonance image to be processed, the image must have the same acquisition parameters as the training dataset, a resolution of not less than 1×1×2, a pixel size of not less than 256×256×64, and the image format must be DICOM format, so that it can be directly input into the generator without additional preprocessing, thus lowering the usage threshold; S53 When the semantic coding branch and the acoustic physical simulation branch process the input image simultaneously, the semantic coding branch quickly extracts the global anatomical structure features of the image according to the preset tile division and embedding rules, and the processing time is controlled to 0. Within 5 seconds, the acoustic physics simulation branch synchronously predicts the physical parameter map and generates a physical prior feature map through a differentiable ray tracking layer, ensuring the temporal consistency of the dual-branch feature output and avoiding feature misalignment. After receiving the feature maps output by the dual branches, the S54 feature fusion decoder first integrates the two types of features through channel splicing, and then gradually restores the image resolution through multiple upsampling convolutional layers. During the upsampling process, a combination of interpolation and convolution is used to avoid loss of feature information. Finally, the number of output channels is adjusted to 1 through a 1×1×1 convolutional kernel to obtain a single-channel virtual ultrasound image. The pixel size of the output image is consistent with the input magnetic resonance image, and the grayscale range is 0-255, which conforms to the observation habits of clinical ultrasound images. Each step is efficiently connected, realizing the rapid and accurate generation from the input image to the virtual ultrasound image, meeting the real-time and practical needs of clinical applications.

[0046] A method for generating virtual ultrasound images from magnetic resonance imaging (MRI) images based on a physics-guided Transformer adversarial network (PTN) is proposed. This method achieves a dual improvement in the physical realism and structural accuracy of the generated virtual ultrasound images through a dual-branch generator architecture. The semantic encoding branch of the generator, utilizing a moving window self-attention mechanism, accurately captures the global anatomical structural features of the MRI image, ensuring the accuracy of the anatomical correspondence between the virtual ultrasound image and the source image. The acoustic physics simulation branch generates physical prior features that conform to acoustic laws by simulating key physical phenomena in the sound wave propagation process, enabling the generated image to naturally exhibit core physical characteristics of real ultrasound images, such as tissue interface reflection and depth attenuation. A dual-domain discriminator constrains the generated image from both spatial and frequency domains. The hybrid objective function, through multi-loss collaborative optimization, further improves the restoration of image texture details, effectively compensating for the shortcomings of traditional methods in terms of physical consistency and feature fusion.

[0047] This method addresses the lack of physical mechanism support in purely data-driven methods by embedding an acoustic physics simulation branch into the generator. This integrates the physical laws of ultrasound imaging into the feature generation process, enabling the output image to no longer merely pursue visual similarity but possess physical properties that conform to the actual acoustic propagation laws, significantly improving the reliability of clinical applications. Furthermore, it addresses the shortcomings of existing methods in feature fusion targeting and the difficulty in balancing anatomical accuracy and texture realism. By designing a dedicated feature fusion decoder, it achieves deep fusion of anatomical semantic features and physical prior features. Simultaneously, with the dual constraints of a dual-domain discriminator, it ensures that the generated image not only fully preserves the precise anatomical structure of the MRI image but also perfectly reproduces the texture details and grayscale distribution characteristics of the real ultrasound image, completely resolving the technical bottleneck of traditional methods that struggle to achieve both simultaneously.

[0048] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0049] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0050] It should be understood that, in various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0051] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0052] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0053] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network, characterized in that, Includes the following steps: S1. Collect three-dimensional magnetic resonance images of the same anatomical site as source domain data and real ultrasound images as target domain data to construct a training dataset; S2, a dual-branch generator network is constructed, which includes a semantic coding branch, an acoustic physics simulation branch, and a feature fusion decoder. The semantic coding branch uses a global feature extraction module based on the Transformer architecture to extract global anatomical features. The acoustic physics simulation branch maps physical parameter maps through a lightweight convolutional network and generates physical prior feature maps through a differentiable ray tracking layer. The feature fusion decoder concatenates the features of the two branches to restore the image resolution. S3, construct a dual-domain discriminator network containing a spatial domain discriminator and a frequency domain discriminator; S4 defines a hybrid objective function that includes adversarial loss, physical consistency loss, and perceptual loss, and performs end-to-end joint training on the generator and discriminator. S5 inputs the magnetic resonance image to be processed into the trained generator, and outputs a virtual ultrasound image for medical diagnosis through dual-branch collaborative processing.

2. The method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network according to claim 1, characterized in that, The reflection coefficient in the acoustic physics simulation branch is calculated using the following formula: ,in, Image coordinates The reflection coefficient at that location, coordinates The corresponding pseudo-acoustic impedance value, coordinates The corresponding pseudo-acoustic impedance value, A small adjustment constant is used to avoid the denominator approaching zero.

3. The method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network according to claim 1, characterized in that, The cumulative attenuation in the acoustic physics simulation branch is calculated using the following formula: ,in, For depth The cumulative attenuation coefficient at that location, coordinates The corresponding tissue attenuation coefficient value, This is the attenuation correction factor. From initial depth to target depth The cumulative index variable.

4. The method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network according to claim 1, characterized in that, The physical prior feature map is generated using the following formula: ,in, coordinates The physical prior eigenvalues ​​at that location, This is the absolute value of the reflection coefficient. This is the cumulative attenuation coefficient. This is a characteristic intensity adjustment factor.

5. The method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network according to claim 1, characterized in that, The physical consistency loss is calculated using the following formula: ,in, This represents the physical consistency loss value. The virtual ultrasound image output by the generator. These are the weighting coefficients for physical prior features. This is a physical prior feature map. This is the grayscale offset. For L1 norm operations, This is the loss balance coefficient.

6. The method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network according to claim 1, characterized in that, S1 includes the following sub-steps: S11, screening three-dimensional magnetic resonance images and real ultrasound images of the same anatomical site to ensure that the images include different tissue types and anatomical structures of the site; S12, performing targeted denoising processing on the screened magnetic resonance and ultrasound images to remove interference signals unrelated to the anatomical structure; S13, performing geometric correction on the denoised images to adjust the image spatial orientation to conform to the preset imaging benchmark; S14, dividing the corrected images into training and validation sets according to a ratio, maintaining the consistency of the anatomical structure distribution among different datasets during the division process to form a complete training dataset.

7. The method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network according to claim 1, characterized in that, S2 includes the following sub-steps: S21, constructing a semantic encoding branch, first dividing the input magnetic resonance image into fixed-size patches and mapping them to linear embedding vectors, then using a Swin Transformer module with a stacked multi-layer moving window self-attention mechanism to calculate the correlation information between patches to extract global anatomical structural features; S22, constructing a parameter prediction subnetwork for the acoustic physics simulation branch, using a two-layer convolutional network to transform the shallow features of the magnetic resonance image, and outputting a pseudo acoustic impedance map and attenuation coefficient map; S23, embedding a differentiable ray tracking layer in the acoustic physics simulation branch, simulating sound wave propagation along a preset scan line direction, and calculating the cumulative attenuation of reflection intensity and depth at different interfaces; S24, constructing a feature fusion decoder, splicing semantic features and physical prior features through the channel dimension, gradually improving the feature map resolution through multiple upsampling convolutional layers, and outputting a virtual ultrasound image.

8. The method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network according to claim 1, characterized in that, S3 includes the following sub-steps: S31, constructing a spatial domain discriminator by building a multi-layer convolutional network using a PatchGAN structure and setting a sliding window to divide the input image into local blocks; S32, the spatial domain discriminator extracts the texture features and grayscale distribution features of each local block through convolution operations and outputs the authenticity judgment result of each local block; S33, constructing a frequency domain discriminator by embedding a fast Fourier transform module to convert the input real ultrasound image and virtual ultrasound image to the frequency domain space; S34, the frequency domain discriminator separates the amplitude spectrum and phase spectrum of the frequency domain image, uses them as dual-channel input to a lightweight convolutional network, and outputs the authenticity judgment result of the frequency domain features through convolution operations.

9. The method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network according to claim 1, characterized in that, S4 includes the following sub-steps: S41, setting the weight coefficients of the hybrid objective function and clarifying the proportions of adversarial loss, physical consistency loss, and perception loss in the total loss; S42, the least squares generative adversarial loss is used to calculate the adversarial loss term, and the outputs of the spatial domain discriminator and the frequency domain discriminator are fused; S43, the deep features of the generated image and the real ultrasound image are extracted using a pre-trained deep network, and the L1 distance between the two is calculated as the perceptual loss term; S44, the loss terms are combined according to the set weight coefficients to form the total loss function, and an alternating training strategy is adopted, first fixing the generator and updating the discriminator parameters, then fixing the discriminator and updating the generator parameters, and iteratively optimizing the network.

10. A method for generating virtual ultrasound images from magnetic resonance images based on a physics-guided Transformer adversarial network according to claim 1, characterized in that, S5 includes the following sub-steps: S51, select the trained generator network, remove the auxiliary training modules used during training, and retain the semantic coding branch, acoustic physical simulation branch, and feature fusion decoder; S52, receive the three-dimensional magnetic resonance image to be processed and input it into the semantic coding branch and acoustic physical simulation branch of the generator; S53, the semantic coding branch and the acoustic physical simulation branch process the input image simultaneously and output the global anatomical feature map and the physical prior feature map respectively; S54, the feature fusion decoder receives the feature maps output by the two branches, and after splicing, upsampling and convolution operations, outputs a virtual ultrasound image corresponding to the input magnetic resonance image.

Images