Training methods, generation methods and equipment for scalp-to-brain magnetic resonance imaging generative models

Abstract

This application provides a training method, generation method, and device for a scalp-to-brain magnetic resonance imaging (MRI) generative model, relating to the field of image processing technology. The training method uses scalp and real brain MRI samples to train a Wasserstein generative adversarial network (GAN) with a frequency domain attention mechanism and gradient penalty. This network includes an encoder, a generator, a feature discriminator, and an image discriminator. The encoder and / or generator integrate a neural network model based on the frequency domain attention mechanism. After training, the encoder and generator together constitute the target generative model. The model parameters are jointly optimized using a composite loss function that includes a frequency domain loss term. This application solves the problems of existing brain image generation technologies ignoring scalp structural information and difficulty in modeling global dependencies and high-frequency details. It achieves high-quality generation of brain MRI data from scalp MRI data, improves the structural integrity and topological fidelity of the generated data, and enhances the reliability of assisted diagnostic applications.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a training method, generation method and device for scalp-to-brain magnetic resonance imaging generative models. Background Technology

[0002] With advancements in medical imaging technology, scalp magnetic resonance imaging (MRI) data is readily available as part of routine scans, and its morphology has an inherent anatomical connection to intracranial brain structures. However, in neuroimaging analysis, scalp data is often discarded as irrelevant information and is not effectively utilized. Therefore, developing a method to reconstruct high-quality brain MRI images from scalp MRI data would not only fully utilize existing data and provide supplementary information for scenarios lacking high-quality brain MRI (such as scans with severe motion artifacts), but also offer a new technical approach to exploring the biological relationship between skull and brain structures, holding significant value for clinical diagnostic support and neuroscience research.

[0003] Currently, existing brain MRI generation techniques are mainly based on generative adversarial networks (GANs) and their variants. These methods have made progress in image synthesis and enhancement, such as generating missing MRI sequences to assist in tumor segmentation. However, existing methods still have significant limitations: First, they often rely on random noise or complete brain images as input, failing to utilize scalp MRI data closely related to brain anatomy, resulting in a lack of reliable structural guidance in the generation process; second, when generating three-dimensional volume data, traditional spatial convolution is difficult to effectively model long-range dependencies between brain regions and generally neglects the use of frequency domain information, leading to insufficient global structural consistency and blurred high-frequency details (such as sulci and gyri) in the generated images. Therefore, it is urgent to address the problems of neglecting scalp structural information and the difficulty in achieving global feature modeling in the frequency domain in existing brain MRI generation techniques. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method for training a scalp-to-brain magnetic resonance imaging generative model, a generation method, and an apparatus to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of this application provides a method for training a scalp-to-brain magnetic resonance imaging generative model, comprising:

[0006] A frequency-domain attention-integrated, gradient-penalized Wasserstein generative adversarial network (GAN) is trained using a training dataset. The training dataset includes multiple sample pairs, each containing a scalp MRI sample and a corresponding real brain MRI sample. The frequency-domain attention-integrated, gradient-penalized Wasserstein GAN comprises: an encoder for receiving the scalp MRI sample and outputting a corresponding latent feature representation; a generator for generating corresponding synthetic brain MRI data based at least on the latent feature representation; a feature discriminator for distinguishing the latent feature representation from random noise vectors sampled from a preset probability distribution; and an image discriminator for distinguishing the synthetic brain MRI data from the corresponding real brain MRI sample. The generator and / or the encoder integrate a neural network module based on a frequency-domain attention mechanism. The encoder and the generator are configured to jointly constitute a scalp-to-brain MRI generative model after training.

[0007] During the training of the fused frequency domain attention mechanism and gradient-penalized Wasserstein generative adversarial network, the encoder, generator, image discriminator and feature discriminator are jointly optimized using a composite loss function that includes a frequency domain loss term.

[0008] In some embodiments of this application, the encoder includes multiple sequentially connected downsampling coding blocks for feature extraction and compression of the scalp magnetic resonance imaging sample, wherein each downsampling coding block includes:

[0009] A first three-dimensional convolutional layer is used to perform convolution operations on the first input features and output downsampled features; wherein, the first input feature processed by the first three-dimensional convolutional layer in the first-level downsampled coding block is the scalp magnetic resonance imaging sample.

[0010] The first activation function layer, connected to the first three-dimensional convolutional layer, is used to perform non-linear activation on the downsampled features and output the first activated features;

[0011] The first 3D instance normalization layer is connected to the first activation function layer and is used to perform sample-by-sample 3D instance normalization processing on the first activated features and output the first normalized features.

[0012] In the downsampling coding block at least one level, after the first three-dimensional instance normalization layer, a neural network module based on a frequency domain attention mechanism is integrated to process the first normalized features to output the first frequency domain self-attention features.

[0013] When the neural network module based on the frequency domain attention mechanism is integrated into the last-level downsampling coding block, the first frequency domain self-attention feature is used as the latent feature representation, and the first normalized feature output by the downsampling coding block that is not the last level is used as the first input feature of the next-level downsampling coding block.

[0014] When the neural network module based on the frequency domain attention mechanism is integrated into a downsampling coding block that is not the last level, the first frequency domain self-attention feature is the first input feature of the next level downsampling coding block, and the first normalized feature output by the last level downsampling coding block is the latent feature representation.

[0015] In some embodiments of this application, the generator includes multiple sequentially connected upsampling generation blocks for upsampling and reconstructing the latent feature representation, wherein each upsampling generation block includes:

[0016] A deconvolutional layer is used to upsample the second input feature and output the upsampled feature; wherein, the second input feature processed by the deconvolutional layer in the first-level upsampling generation block is the latent feature representation, or is a concatenation feature of the latent feature representation and random noise;

[0017] The second three-dimensional convolutional layer is connected to the deconvolutional layer and is used to perform convolution operations on the upsampled features to output convolutional features;

[0018] The second 3D instance normalization layer is connected to the second 3D convolutional layer and is used to perform feature-by-feature 3D instance normalization processing on the convolutional features and output the second normalized features.

[0019] The second activation function layer, connected to the second 3D instance normalization layer, is used to perform non-linear activation on the second normalized feature and output the second activated feature.

[0020] In the at least one level of the upsampling generation block, after the second activation function layer, a neural network module based on a frequency domain attention mechanism is integrated to process the second activated features to output the second frequency domain self-attention features.

[0021] The generator also includes:

[0022] The frequency domain sharpening module, connected after the last-stage upsampling generation block, is used to perform frequency domain self-attention enhancement processing on the received features and output sharpened features.

[0023] An output mapping layer, connected to the frequency domain sharpening module, is used to perform channel mapping and value domain transformation on the sharpening features and output the synthesized brain magnetic resonance imaging data.

[0024] Wherein, when the neural network module based on the frequency domain attention mechanism is integrated into the last-level upsampling generation block, the neural network module based on the frequency domain attention mechanism passes the second frequency domain self-attention feature to the frequency domain sharpening module, and the second activated feature output by the upsampling generation block that is not the last level is used as the second input feature of the next level upsampling generation block;

[0025] When the neural network module based on the frequency domain attention mechanism is integrated into the upsampling generation block that is not the last level, the second frequency domain self-attention feature is used as the second input feature of the next level upsampling generation block, and the second activated feature output by the last level upsampling generation block is passed to the frequency domain sharpening module.

[0026] In some embodiments of this application, the neural network module based on the frequency domain attention mechanism includes a three-dimensional frequency domain self-attention module, which includes:

[0027] The third 3D instance normalization layer is used to perform feature-by-feature 3D instance normalization processing on the third input feature and output the third normalized feature; wherein, when the 3D frequency domain self-attention module is integrated into the encoder, the third input feature is the first normalized feature output by the previous level downsampling coding block; when the 3D frequency domain self-attention module is integrated into the generator, the third input feature is the second activated feature output by the previous level upsampling generation block;

[0028] A parallel branch processing unit is connected to the third 3D instance normalization layer. The parallel branch processing unit includes: a first branch for processing the third normalization feature to generate a query matrix, a second branch for processing the third normalization feature to generate a key matrix, and a third branch for processing the third normalization feature to generate a value matrix.

[0029] The first Fast Fourier Transform unit, connected to the parallel branch processing unit, is used to transform the query matrix and the key matrix from the spatial domain to the frequency domain;

[0030] A frequency domain attention calculation unit, connected to the first fast Fourier transform unit, is used to calculate attention weights in the frequency domain based on the query matrix and the key matrix, and to use the attention weights to perform weighted fusion on the value matrix to output frequency domain weighted features.

[0031] The first inverse fast Fourier transform unit is connected to the frequency domain attention calculation unit and is used to transform the frequency domain weighted features from the frequency domain to the spatial domain and output the spatial domain features.

[0032] An output adjustment layer, connected to the first inverse fast Fourier transform unit, is used to adjust the channels of the spatial domain features, output a first self-attention feature, and add the first self-attention feature and the third input feature through a residual connection to form a first target frequency domain self-attention feature; wherein, when the three-dimensional frequency domain self-attention module is integrated into the encoder, the first target frequency domain self-attention feature is the first frequency domain self-attention feature; when the three-dimensional frequency domain self-attention module is integrated into the generator, the first target frequency domain self-attention feature is the second frequency domain self-attention feature.

[0033] In some embodiments of this application, the neural network module based on the frequency domain attention mechanism includes a three-dimensional frequency domain feedforward network module, which includes:

[0034] The fourth 3D instance normalization layer is used to perform feature-by-feature 3D instance normalization processing on the fourth input feature and output the fourth normalized feature; wherein, when the 3D frequency domain feedforward network module is integrated into the encoder, the fourth input feature is the first normalized feature output by the previous level downsampling coding block; when the 3D frequency domain feedforward network module is integrated into the generator, the fourth input feature is the second activated feature output by the previous level upsampling generation block;

[0035] The channel transformation layer, connected to the fourth 3D instance normalization layer, is used to transform the fourth normalized features in terms of channel dimensions and output features in the form of 3D tensors.

[0036] An image patch serialization unit, connected to the channel transformation layer, is used to unfold the features in the form of a three-dimensional tensor into an image patch sequence;

[0037] The second Fast Fourier Transform unit is connected to the image block serialization unit and is used to convert the image block sequence from the spatial domain to the frequency domain.

[0038] A learnable frequency domain modulation unit is connected to the second fast Fourier transform unit, which is used to multiply the transformed features element-wise with a learnable weight matrix in the frequency domain and output the modulated spectrum.

[0039] The second inverse fast Fourier transform unit is connected to the learnable frequency domain modulation unit and is used to convert the modulated spectrum from the frequency domain back to the spatial domain and output a spatial domain sequence.

[0040] An image patch reconstruction unit, connected to the second inverse fast Fourier transform unit, is used to reconstruct the spatial domain sequence into reconstructed features in the form of a three-dimensional tensor.

[0041] A gated activation layer, connected to the image patch reconstruction unit, is used to perform nonlinear activation on the reconstruction features in the form of three-dimensional tensors, outputting a first feedforward activation feature, and adding the first feedforward activation feature and the fourth input feature through a residual connection to form a second target frequency domain feedforward feature; wherein, when the three-dimensional frequency domain feedforward network module is integrated into the encoder, the second target frequency domain feedforward feature is the first frequency domain feedforward feature; when the three-dimensional frequency domain feedforward network module is integrated into the generator, the second target frequency domain feedforward feature is the second frequency domain feedforward feature.

[0042] In some embodiments of this application, the composite loss function is a weighted sum of multiple losses, including:

[0043] The first adversarial loss term is calculated based on the image discriminator's distinction between the synthetic brain MRI data and the real brain MRI sample, and is used to optimize the generator.

[0044] The second adversarial loss term is calculated based on the distinction result between the latent feature representation and the random noise vector by the feature discriminator, and is used to optimize the encoder;

[0045] The frequency domain loss term is obtained by converting the synthetic brain magnetic resonance imaging data and the real brain magnetic resonance imaging sample to the frequency domain and calculating the difference in frequency domain representation.

[0046] The multi-scale spatial loss term is obtained by calculating the absolute error between the synthetic brain MRI data and the real brain MRI sample on image patches of different scales;

[0047] The perceptual similarity loss term is obtained by comparing the difference between the depth feature representations of the synthetic brain magnetic resonance imaging data and the real brain magnetic resonance imaging samples.

[0048] The feature matching loss term is obtained by calculating the difference between the features extracted by the intermediate multi-layer network layers of the encoder from the scalp magnetic resonance imaging sample and the synthetic brain magnetic resonance imaging data.

[0049] In some embodiments of this application, the feature discriminator includes multiple fully connected layers for processing the latent feature representation and random noise vectors sampled from the preset probability distribution and outputting a first scalar score. The first scalar score is used to construct a second adversarial loss term in the composite loss function to constrain the distribution of the latent feature representation to be close to the preset probability distribution.

[0050] Each fully connected layer of the feature discriminator is sequentially connected to a fifth 3D instance normalization layer and a fifth activation function layer.

[0051] In some embodiments of this application, the image discriminator is a discriminator based on a three-dimensional convolutional neural network, used to process the synthetic brain magnetic resonance imaging data and the real brain magnetic resonance imaging samples and output a second scalar score. The second scalar score is used to construct the first adversarial loss term in the composite loss function, such that the first adversarial loss term is calculated based on the second scalar score, and a gradient penalty is introduced during the calculation process to follow the Wasserstein distance optimization criterion.

[0052] Another aspect of this application provides a method for generating scalp-to-brain magnetic resonance images, comprising:

[0053] Acquire scalp magnetic resonance imaging data of the target object;

[0054] The scalp magnetic resonance imaging data is input into the scalp-to-brain magnetic resonance imaging generation model, so that the scalp-to-brain magnetic resonance imaging generation model outputs corresponding synthetic brain magnetic resonance imaging data, wherein the scalp-to-brain magnetic resonance imaging generation model is trained based on the scalp-to-brain magnetic resonance imaging generation model training method provided in the first aspect above.

[0055] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the scalp-to-brain magnetic resonance imaging generation model training method, and / or the scalp-to-brain magnetic resonance imaging generation method.

[0056] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the scalp-to-brain magnetic resonance imaging generative model training method, and / or the scalp-to-brain magnetic resonance imaging generative method.

[0057] A fifth aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, implements the scalp-to-brain magnetic resonance imaging (MRI) generation model training method, and / or the scalp-to-brain MRI generation method.

[0058] The scalp-to-brain magnetic resonance imaging (MRI) generative model training method provided in this application, by using scalp MRI samples as input to the generator, constructs for the first time a mapping relationship between the scalp and the brain. This gives the generation process a clear anatomical basis, overcoming the limitations of traditional methods that rely on random noise or unrelated source images, and significantly improving the biological rationality and clinical applicability of the generated images. By integrating a neural network module based on a frequency domain attention mechanism into the encoder and / or generator, and using a composite loss function including a frequency domain loss term for joint supervision, dual-domain collaborative modeling of the frequency and spatial domains can be achieved. This allows the network to efficiently model the global long-range dependencies of brain structures in the frequency domain, overcoming the fundamental defect of the limited receptive field of traditional spatial convolution, thereby significantly improving the structural integrity and topological fidelity of the generated images. The introduction of frequency domain attention mechanisms and frequency domain loss terms enables the model to specifically enhance and constrain high-frequency components in images (such as brain sulci, cortical texture, and white matter boundaries). This effectively alleviates the detail blurring problem caused by excessive smoothing in existing methods, thereby improving the clarity and anatomical accuracy of the generated images and enhancing their usability in assisted diagnosis. By employing a Wasserstein Generative Adversarial Network (WGAN-GP) framework with gradient penalty and explicitly defining a four-component structure including an encoder, generator, feature discriminator, and image discriminator, a complete, end-to-end trainable generative system is formed. This framework features stable training and strong scalability, providing the first systematic technical implementation path for the novel task of scalp-driven brain image generation, laying the foundation for its application in data augmentation, image completion, and other scenarios.

[0059] Additional advantages, objectives, and features of this application will be set forth in part in the description which follows, and will in part become apparent to those skilled in the art upon review of the following description, or may be learned by practice of the application. The objectives and other advantages of this application can be realized and obtained by means of the structures specifically pointed out in the specification and drawings.

[0060] Those skilled in the art will understand that the purposes and advantages that can be achieved with this application are not limited to those specifically described above, and that the above and other purposes that this application can achieve will be more clearly understood from the following detailed description. Attached Figure Description

[0061] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, do not constitute a limitation thereof. The components in the drawings are not drawn to scale but are merely for illustrating the principles of this application. For ease of illustration and description of certain parts of this application, corresponding portions in the drawings may be enlarged, i.e., may appear larger relative to other components in an exemplary device actually manufactured according to this application. In the drawings:

[0062] Figure 1 This is a schematic flowchart of a scalp-to-brain magnetic resonance imaging generative model training method in one embodiment of this application.

[0063] Figure 2 This is a schematic diagram illustrating an example architecture of a Wasserstein generative adversarial network with a fused frequency domain attention mechanism and gradient penalty, as described in one embodiment of this application.

[0064] Figure 3 This is a schematic diagram illustrating an example of the architecture of a downsampling coding block in one embodiment of this application.

[0065] Figure 4 This is a schematic diagram illustrating an example of the architecture of an upsampling generation block in one embodiment of this application.

[0066] Figure 5 This is a schematic diagram illustrating an example of the architecture of a three-dimensional frequency domain self-attention module in one embodiment of this application.

[0067] Figure 6 This is a schematic diagram illustrating the architecture of a three-dimensional frequency domain feedforward network module in one embodiment of this application.

[0068] Figure 7 This is a schematic flowchart of a method for generating scalp-to-brain magnetic resonance imaging in one embodiment of this application.

[0069] Figure 8 This is a schematic diagram of the generative network framework for a method of generating high-quality three-dimensional brain MRI images based on scalp MRI data, as shown in an application example of this application.

[0070] Figure 9 This is a schematic diagram illustrating the architecture of a 3D frequency domain self-attention module in an application example of this application.

[0071] Figure 10 This is a schematic diagram illustrating the architecture of a 3D frequency domain feedforward network module in an application example of this application.

[0072] Figure 11 This is a schematic diagram comparing real brain magnetic resonance imaging data with the generated results of each model.

[0073] Figure 12(a) is a schematic diagram comparing the numerical distribution of synthetic brain magnetic resonance imaging data and real brain magnetic resonance imaging data using a 3D variational autoencoder.

[0074] Figure 12(b) is a schematic diagram comparing the numerical distribution of synthetic brain magnetic resonance imaging data and real brain magnetic resonance imaging data of the model of this application with the introduction of frequency domain network module.

[0075] Figure 12(c) is a schematic diagram comparing the numerical distribution of synthetic brain magnetic resonance imaging data and real brain magnetic resonance imaging data using a 3D encoder generative adversarial network.

[0076] Figure 12(d) is a schematic diagram comparing the numerical distribution of synthetic brain MRI data from a 3D α-adversarial generative network with that of real brain MRI data.

[0077] Figure 12(e) is a schematic diagram comparing the numerical distribution of synthetic brain magnetic resonance imaging data and real brain magnetic resonance imaging data using the batch normalized model of this application.

[0078] Figure 12(f) is a schematic diagram comparing the numerical distribution of synthetic brain magnetic resonance imaging data and real brain magnetic resonance imaging data using the instance-normalized model of this application.

[0079] Figure 13(a) is a schematic diagram comparing the relationship between synthetic brain magnetic resonance imaging data generated by a 3D variational autoencoder on the test set and real brain magnetic resonance imaging data.

[0080] Figure 13(b) is a schematic diagram comparing the relationship between synthetic brain magnetic resonance imaging data and real brain magnetic resonance imaging data generated on the test set by the model of this application with the introduction of frequency domain network module.

[0081] Figure 13(c) is a schematic diagram comparing the relationship between synthetic brain MRI data generated by the 3D encoder generative adversarial network on the test set and real brain MRI data.

[0082] Figure 13(d) is a schematic diagram comparing the relationship between synthetic brain MRI data generated by the 3D α-adversarial generative network on the test set and real brain MRI data.

[0083] Figure 13(e) is a schematic diagram comparing the relationship between synthetic brain magnetic resonance imaging data and real brain magnetic resonance imaging data generated on the test set using the batch normalized model of this application.

[0084] Figure 13(f) is a schematic diagram comparing the relationship between synthetic brain magnetic resonance imaging data and real brain magnetic resonance imaging data generated on the test set using the application model with instance normalization. Detailed Implementation

[0085] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain this application, but are not intended to limit it.

[0086] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the structures and / or processing steps closely related to the solution according to this application are shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0087] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0088] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0089] In the following description, embodiments of the present application will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0090] First, it's important to note that current methods for generating brain MRI images are primarily based on Generative Adversarial Networks (GANs) and their variants. Studies have shown that GANs can synthesize missing T1-weighted sequences and Fluid Attenuated Inversion Recovery (FLAIR) sequences from MRI images, achieving effective brain tumor segmentation. Early studies used the original Vanilla GAN to generate brain MRI images, but the generated images had low resolution and insufficient anatomical accuracy. Subsequent research introduced Conditional Generative Adversarial Networks (cGANs) and progressive training strategies such as Progressive Growing of Generative Adversarial Networks (PGGANs) to improve the realism and contextual accuracy of the generated images. However, these methods still have the following limitations:

[0091] (1) Relying on random noise or simple source images as input ignores skull data related to brain structure, which limits the expressive power of the model and may miss the key relationship between skull and brain tissue.

[0092] (2) Existing research mainly focuses on two-dimensional image generation, while the generation of three-dimensional volumetric images has progressed relatively slowly due to limitations in computational complexity and data requirements.

[0093] (3) Insufficient utilization of frequency domain information: Although frequency domain analysis performs well in image segmentation and enhancement tasks, it has not been fully explored in 3D brain MRI generation, especially in modeling global dependencies of brain structures and high-frequency details.

[0094] (4) Weak ability to preserve individual characteristics: Traditional normalization methods such as batch normalization (BatchNorm) are prone to losing the differences between individuals when processing small batches or individual samples, which affects the personalized expression ability of the generated images and limits their application in clinical personalized analysis.

[0095] Therefore, developing a method to generate high-quality brain MRI images from skull data such as scalp MRI images would not only fill a key gap in the field of medical image generation, but also provide a new technical approach for exploring the relationship between skull and brain structures, which is of great value for clinical auxiliary diagnosis and neuroscience research.

[0096] However, there is currently a lack of paradigms for generating brain images using the scalp as input. Specifically, existing methods for generating 3D brain MRI images mostly rely on random noise or complete images as input, failing to effectively utilize the anatomical information contained in the scalp data, resulting in a lack of anatomical consistency between the generated images and the actual brain structures. Similarly, there is a lack of standardized generation frameworks suitable for scalp-brain mapping. Specifically, there is currently no systematic technical solution to support the standardized generation of brain structure images from scalp MRI, limiting the application of this technology in scenarios such as data augmentation, image completion, and cross-modal inference. At the same time, traditional spatial convolution is difficult to model the global dependencies of brain structures. Specifically, brain structures have complex spatial correlations and long-range dependencies, and traditional convolution operations are limited by local receptive fields, making it difficult to effectively capture the global connections between key structures such as sulci and gyri, and white matter boundaries.

[0097] Based on this, in order to address the problems of existing brain image generation technologies such as ignoring scalp structural information and difficulty in modeling global dependencies and high-frequency details, embodiments of this application provide a scalp-to-brain magnetic resonance imaging (MRI) generation model training method, a scalp-to-brain MRI generation model training device for executing the scalp-to-brain MRI generation model training method, a scalp-to-brain MRI generation method, a scalp-to-brain MRI generation device for executing the scalp-to-brain MRI generation method, an electronic device, a computer-readable storage medium, and a computer program product. These technologies can achieve high-resolution generation of brain MRI data from scalp MRI data, improve the structural integrity and topological fidelity of the generated data, and enhance the reliability of the generated image-assisted diagnosis application.

[0098] The following examples will provide a detailed description.

[0099] Based on this, embodiments of this application provide a method for training a scalp-to-brain magnetic resonance imaging (MRI) generative model, which can be implemented by a scalp-to-brain MRI generative model training device. See [link to relevant documentation]. Figure 1The scalp-to-brain magnetic resonance imaging generative model training method specifically includes the following:

[0100] Step 100: Train a frequency-domain attention-based and gradient-penalized Wasserstein generative adversarial network using a training dataset. The training dataset includes multiple sample pairs, each containing a scalp MRI sample and a corresponding real brain MRI sample. The frequency-domain attention-based and gradient-penalized Wasserstein generative adversarial network includes: an encoder for receiving the scalp MRI sample and outputting a corresponding latent feature representation; a generator for generating corresponding synthetic brain MRI data based at least on the latent feature representation; a feature discriminator for distinguishing the latent feature representation from random noise vectors sampled from a preset probability distribution; and an image discriminator for distinguishing the synthetic brain MRI data from the corresponding real brain MRI sample. The generator and / or the encoder integrate a neural network module based on a frequency-domain attention mechanism. The encoder and the generator are configured to jointly form a scalp-to-brain MRI generative model after training.

[0101] In one or more embodiments of this application, the scalp magnetic resonance imaging sample can be abbreviated as scalp MRI sample. The real brain magnetic resonance imaging sample can be abbreviated as real MIR sample; the synthetic brain magnetic resonance imaging data can be written as synthetic MRI data. The scalp magnetic resonance imaging sample is image data acquired by a magnetic resonance imaging device, primarily displaying the scalp, skull, and other peripheral head tissues. The real brain magnetic resonance imaging sample is high-quality image data acquired by a magnetic resonance imaging device, displaying the cerebral cortex, white matter, cerebrospinal fluid, and other intracranial brain tissues; it is the target that the model of this application aims to generate and also serves as the supervision signal during training. The synthetic brain magnetic resonance imaging data is image data simulating the real brain anatomy structure calculated by the generator based on the input latent feature representation; it is the object compared with the real brain magnetic resonance imaging sample during training and is also the application output of the scalp-to-brain magnetic resonance imaging generation model obtained after training.

[0102] Understandably, before step 100, raw scalp MRI data and raw real brain MRI data for multiple imaging subjects (such as patients or volunteers) can be obtained from public datasets for model training. Then, the raw scalp MRI data and raw real brain MRI data for each imaging subject can be directly used as a scalp MRI sample and a real brain MRI sample in a sample pair. To further improve the reliability and stability of model training, the raw scalp MRI data and raw real brain MRI data can be preprocessed (e.g., normalizing the intensity of the raw scalp MRI data and raw real brain MRI data to [-1,1], cropping to 64×64×64 voxels, and removing all-zero slices). The preprocessed scalp MRI data and real brain MRI data can then be used as scalp MRI samples and real brain MRI samples, respectively.

[0103] It should be noted that the Frequency Domain Attention Wasserstein Generative Adversarial Network with Gradient Penalty (FDA-WGAN-GP) is a three-dimensional magnetic resonance imaging (3D MRI) data generation framework proposed for the first time in this application, which generates data from the scalp to the brain (i.e., obtaining synthetic brain MRI data based on scalp MRI data). It introduces gradient penalty (GP) on the basis of Wasserstein Generative Adversarial Network (WGAN) to stabilize training; and integrates a frequency domain attention mechanism into the network to model the global dependencies of features. Through joint modeling of frequency domain and spatial domain and individual feature preservation strategy, high-fidelity brain image synthesis is achieved. The frequency domain attention washerstein generative adversarial network with gradient penalty can be simply referred to as the FDA-WGAN-GP model.

[0104] See Figure 2 The fused frequency domain attention mechanism and gradient-penalized Wasserstein generative adversarial network includes:

[0105] (1) Encoder 10 is used to receive the scalp magnetic resonance imaging sample and output the corresponding latent feature representation; specifically, encoder 10 receives and processes the scalp magnetic resonance imaging sample, extracts and compresses its deep features through multi-layer downsampling convolution and other operations, and finally outputs a low-dimensional latent feature representation (i.e., latent code z), which is the starting point for realizing the mapping from scalp to brain.

[0106] The latent feature representation is a low-dimensional, dense feature vector or feature tensor output by the encoder 10 after encoding the scalp magnetic resonance imaging sample. The latent feature representation is an abstract summary of scalp anatomical information and serves as the core conditional information for the generator 20 to reconstruct the brain image. In one example, the dimension of the latent feature representation can be 512×4×4×4.

[0107] (2) Generator 20, whose input is connected to the output of encoder 10 and whose output is connected to image discriminator 40, generator 20 is used to generate corresponding synthetic brain magnetic resonance imaging data based at least on the latent feature representation output by encoder 10; specifically, generator 20 uses the latent feature representation output by encoder 10 as a condition (or input) and reconstructs and generates corresponding synthetic brain magnetic resonance imaging data through multi-layer upsampling and deconvolution operations.

[0108] (3) Feature discriminator 30, whose input is connected to the output of encoder 10, is used to distinguish between the latent feature representation output by encoder 10 and random noise vector sampled from a preset probability distribution. Specifically, feature discriminator 30 receives and distinguishes two types of input: one is the latent feature representation output by encoder 10, and the other is the random noise vector sampled from a preset probability distribution (such as a standard normal distribution). Its goal is to constrain the distribution of the latent feature representation to be close to the preset distribution, thereby stabilizing training and improving generation quality.

[0109] The preset probability distribution is a probability distribution predefined before training, used to sample random noise vectors as a comparison benchmark for the feature discriminator 30, and can also serve as an optional additional input for the generator 20. In one example, the preset probability distribution can be a standard normal distribution N(0, 1).

[0110] (4) An image discriminator 40, whose input is connected to the output of the generator 20, is used to distinguish between the synthetic brain MRI data output by the generator 20 and the real brain MRI samples that belong to the same sample pair as the scalp MRI samples corresponding to the synthetic brain MRI data. Specifically, the image discriminator 40 receives and distinguishes two types of input: one is the brain MRI data synthesized by the generator 20, and the other is the corresponding real brain MRI samples in the training dataset. Under the optimization objective of Wasserstein Distance with Gradient Penalty, the image discriminator 40 drives the generator 20 to generate more realistic images. In one example, the parameter of the gradient penalty can be written as The value can be 10.

[0111] At least one of the generator 20 and the encoder 10 integrates a neural network module 50 based on a frequency domain attention mechanism, in such a way as... Figure 2 In one example, both the generator 20 and the encoder 10 integrate a neural network module 50 based on a frequency domain attention mechanism. This frequency domain attention-based neural network module 50 is a trainable neural network substructure integrated into the encoder or generator. It performs attention weight calculation or feature modulation by transforming features to the frequency domain. It uses Fast Fourier Transform (FFT) to transform features to the frequency domain for computation, efficiently modeling global context relationships and enhancing high-frequency details, achieving dual-domain collaboration between the frequency and spatial domains. Preferred implementations include a frequency domain self-attention module (FDSA3D) and a frequency domain feedforward network module (FDFFN3D), which will be described in detail in subsequent embodiments.

[0112] Step 200: During the training of the fused frequency domain attention mechanism and gradient-penalized Wasserstein generative adversarial network, the encoder, generator, image discriminator and feature discriminator are jointly optimized using a composite loss function that includes a frequency domain loss term.

[0113] In step 200, the composite loss function including the frequency domain loss term is a multi-objective loss function for jointly optimizing the encoder, generator, image discriminator, and feature discriminator. That is, it refers to the weighted sum of multiple losses. The frequency domain loss term specifically refers to the loss calculated in the frequency domain to account for the difference between synthetic brain MRI data and real brain MRI samples; in this embodiment, it can also be called asymptotic frequency domain loss. This composite loss is crucial to ensuring that the model achieves its optimization objectives across multiple dimensions (such as adversarial, frequency domain consistency, multi-scale similarity, and perceptual similarity).

[0114] The joint optimization refers to using the aforementioned composite loss function during training to simultaneously or alternately update the parameters of encoder 10, generator 20, feature discriminator 30, and image discriminator 40, enabling the entire generative adversarial network framework to learn collaboratively, ultimately allowing generator 20 to generate high-quality synthetic brain magnetic resonance imaging data from scalp magnetic resonance imaging data.

[0115] As described above, the scalp-to-brain magnetic resonance imaging (MRI) generative model training method provided in this application constructs a scalp-to-brain mapping relationship for the first time by using scalp MRI samples as input to the generator. This gives the generation process a clear anatomical basis, overcoming the limitations of traditional methods that rely on random noise or unrelated source images, and significantly improving the biological rationality and clinical applicability of the generated images. By integrating a neural network module based on a frequency domain attention mechanism into the encoder and / or generator, and using a composite loss function containing a frequency domain loss term for joint supervision, dual-domain collaborative modeling of the frequency and spatial domains can be achieved. This allows the network to efficiently model the global long-range dependencies of brain structures in the frequency domain, overcoming the fundamental defect of limited receptive fields in traditional spatial convolution, thereby significantly improving the structural integrity and topological fidelity of the generated images. The introduction of frequency domain attention mechanisms and frequency domain loss terms enables the model to specifically enhance and constrain high-frequency components (such as sulci, cortical texture, and white matter boundaries) in scalp MRI data. This effectively alleviates the detail blurring problem caused by excessive smoothing in existing methods, thereby improving the clarity and anatomical accuracy of the generated images and enhancing their usability in assisted diagnosis. By employing a Wasserstein Generative Adversarial Network (WGAN-GP) framework with gradient penalty and explicitly defining a four-component structure including an encoder, generator, feature discriminator, and image discriminator, a complete, end-to-end trainable generative system is formed. This framework features stable training and strong scalability, providing the first systematic technical implementation path for the novel task of scalp-driven brain image generation, laying the foundation for its application in data augmentation, image completion, and other scenarios.

[0116] To further address the challenges of efficiently and structurally extracting and compressing scalp features in the encoder, enhancing the personalized expressiveness of the generated image, and ensuring seamless integration of the frequency domain attention module with the encoding process, this application provides a scalp-to-brain magnetic resonance imaging (MRI) generative model training method. In this method, the encoder 10 includes multiple sequentially connected downsampling coding blocks 60 for feature extraction and compression of the scalp MRI samples. In one example, the encoder 10 may contain four sequentially connected downsampling coding blocks 60.

[0117] Among them, see Figure 3 Each downsampling coding block 60 specifically contains the following:

[0118] (1) A first three-dimensional convolutional layer 61 is used to perform convolution operation on the first input feature and output downsampled features; wherein, the first input feature processed by the first three-dimensional convolutional layer 61 in the first-level downsampled coding block 60 is the scalp magnetic resonance imaging sample;

[0119] (2) The first activation function layer 62 is connected to the first three-dimensional convolutional layer 61 and is used to perform non-linear activation on the downsampled features and output the first activated features;

[0120] (3) The first three-dimensional instance normalization layer 63 is connected to the first activation function layer 62 and is used to perform sample-by-sample three-dimensional instance normalization processing on the first activated features and output the first normalized features.

[0121] (4) In at least one level of the downsampling coding block 60, after the first three-dimensional instance normalization layer 63, a neural network module 50 based on the frequency domain attention mechanism is integrated to process the first normalized feature to output the first frequency domain self-attention feature; wherein, the first frequency domain self-attention feature specifically refers to the feature output by the integrated neural network module 50 based on the frequency domain attention mechanism in the encoder 10 after processing the first normalized feature, which is the feature representation after frequency domain attention enhancement.

[0122] When the neural network module 50 based on the frequency domain attention mechanism is integrated into the last-level downsampling coding block 60, the first frequency domain self-attention feature is used as the latent feature representation, and the first normalized feature output by the non-last-level downsampling coding block 60 is used as the first input feature of the next-level downsampling coding block 60.

[0123] When the neural network module 50 based on the frequency domain attention mechanism is integrated into the downsampling coding block 60 which is not the last level, the first frequency domain self-attention feature is the first input feature of the next level downsampling coding block 60, and the first normalized feature output by the last level downsampling coding block 60 is the latent feature representation.

[0124] exist Figure 3 In this example, the downsampling coding block 60 is equipped with a neural network module 50 based on the frequency domain attention mechanism.

[0125] Specifically, the downsampling coding block 60 refers to the core repetitive unit within the encoder 10. Multi-level indicates that this structure is repeated multiple times (e.g., 4 levels), with each level processing features at different resolutions or levels of abstraction. Sequential connection means that the output of the previous level serves as the input of the next level, forming a chain-like data flow. The downsampling coding block emphasizes that each unit performs the function of reducing spatial resolution (downsampling) and extracting deeper features (encoding). The first 3D convolutional layer 61, the first activation function layer 62, and the first 3D instance normalization layer 63 are the basic operational layers constituting the downsampling coding block 60. The prefix "first" is used to uniquely identify these layers located in the encoder, distinguishing them from corresponding layers in other components such as the generator.

[0126] As described above, the scalp-to-brain magnetic resonance imaging (MRI) generative model training method provided in this application, through a standardized hierarchical design of sequentially connected downsampling coding blocks, ensures a stable and repeatable extraction and compression process of features from detail to abstraction and from high dimension to low dimension, providing the generator with high-quality, structured conditional input. By forcibly using a first three-dimensional instance normalization layer in each coding block, each sample is independently normalized at the initial stage of feature extraction. This effectively preserves the unique feature distribution of individual samples from the source, thereby suppressing the smoothing of individual information by batch effects. This provides a crucial prerequisite for generating brain images with personalized anatomical features, further addressing the problem that traditional normalization methods (such as BatchNorm) average individual features during batch processing, leading to template-like generated images that fail to reflect the anatomical variability of different subjects, resulting in the loss of individual differences during generation.

[0127] To further address the specific architectural challenge of how to reconstruct high-quality, highly personalized, and high-fidelity 3D brain images with clear anatomical details from low-dimensional latent features within the generator, this application provides a scalp-to-brain magnetic resonance imaging (MRI) generative model training method. The generator 20 includes multi-level sequentially connected upsampling generation blocks 70 for upsampling and reconstructing the latent feature representation. In one example, the generator 20 may contain six levels of sequentially connected upsampling generation blocks 70. Specifically, the core function of the upsampling generation blocks is to increase the spatial size of the feature map (upsampling) and perform feature reconstruction (generation).

[0128] Among them, see Figure 4 Each upsampling generation block 70 specifically contains the following:

[0129] (1) Deconvolution layer 71 is used to upsample the second input feature and output the upsampled feature; wherein, the second input feature processed by the deconvolution layer 71 in the first-level upsampling generation block 70 is the latent feature representation, or a concatenation feature of the latent feature representation and a random noise. Specifically, the deconvolution layer 71 can also be called a transposed convolution layer, which is a key operation layer for realizing feature map upsampling. Through a learnable convolution kernel, it performs the opposite operation to regular convolution, thereby increasing the feature map space size while performing feature transformation. The concatenation feature refers to connecting the latent feature representation and the random noise vector together in a certain dimension (usually the channel dimension or concatenated after flattening) to form a new, higher-dimensional combined feature.

[0130] (2) The second three-dimensional convolutional layer 72 is connected to the deconvolutional layer 71 and is used to perform convolution operation on the upsampled features and output convolutional features;

[0131] (3) The second three-dimensional instance normalization layer 73 is connected to the second three-dimensional convolutional layer 72 and is used to perform feature-by-feature three-dimensional instance normalization processing on the convolutional features and output the second normalized features.

[0132] (4) The second activation function layer 74 is connected to the second three-dimensional instance normalization layer 73 and is used to perform nonlinear activation on the second normalized feature and output the second activated feature;

[0133] (5) In the at least one level of the upsampling generation block 70, after the second activation function layer 74, a neural network module 50 based on the frequency domain attention mechanism is integrated to process the second activated features to output the second frequency domain self-attention features; specifically, the second frequency domain self-attention features refer to the features output by the integrated neural network module 50 based on the frequency domain attention mechanism in the generator 20 after processing the second activated features, which are intermediate features optimized by frequency domain attention and used for subsequent reconstruction or sharpening.

[0134] The generator 20 also specifically includes the following content:

[0135] The frequency domain sharpening module 80, connected after the last-stage upsampling generation block 70, is used to perform frequency domain self-attention enhancement processing on the received features and output sharpened features. Specifically, the frequency domain sharpening module 80 refers to an independent module connected to the end of the generator backbone network (upsampling generation block sequence). Its function is to perform additional and enhanced frequency domain self-attention processing on the features that have already undergone spatial upsampling, in order to focus on enhancing the high-frequency details and edge sharpness of the final image.

[0136] The output mapping layer 90, connected to the frequency domain sharpening module 80, is used to perform channel mapping and value range transformation on the sharpening features, and output the synthesized brain magnetic resonance imaging data. Specifically, the output mapping layer 90 refers to the final layer of the generator 20, which typically consists of a 1×1×1 convolutional layer and a Tanh activation function. Its function is to map the number of channels of the sharpening features to the target number of output channels (e.g., a grayscale image with one channel) and transform the pixel values ​​to a specified range (e.g., [-1,1]), thereby obtaining the final synthesized brain magnetic resonance imaging data.

[0137] When the neural network module 50 based on the frequency domain attention mechanism is integrated into the last-level upsampling generation block 70, the neural network module 50 based on the frequency domain attention mechanism passes the second frequency domain self-attention feature to the frequency domain sharpening module 80, and the second activated feature output by the upsampling generation block 70 which is not the last level is used as the second input feature of the next level upsampling generation block 70.

[0138] When the neural network module 50 based on the frequency domain attention mechanism is integrated into the upsampling generation block 70 which is not the last level, the second frequency domain self-attention feature is used as the second input feature of the next level upsampling generation block 70, and the second activated feature output by the last level upsampling generation block 70 is passed to the frequency domain sharpening module 80.

[0139] exist Figure 4 In this example, the upsampling generation block 70 is equipped with a neural network module 50 based on the frequency domain attention mechanism, and the current upsampling generation block 70 is the last-level upsampling generation block. The neural network module 50 based on the frequency domain attention mechanism in the upsampling generation block 70 is connected to the frequency domain sharpening module 80.

[0140] As described above, the scalp-to-brain magnetic resonance imaging (MRI) generation model training method provided in this application achieves a controllable and progressive reconstruction process from low-dimensional features to high-resolution images through a standardized design of multi-level sequentially connected upsampling generation blocks. Each level, through a fixed process of deconvolution upsampling, convolutional thinning, instance normalization to activation, gradually enriches the feature content while increasing the spatial size, laying the foundation for generating structurally reasonable images. By integrating a frequency domain attention module in at least one upsampling block, frequency domain global modeling is performed in the intermediate feature layer to optimize the feature representation being reconstructed. By setting a dedicated frequency domain sharpening module at the end of the reconstruction path, a final, more targeted frequency domain self-attention enhancement is performed on the features that are about to be mapped to pixel values, specifically focusing on improving the high-frequency clarity and structural sharpness of the final output image, thereby significantly improving the visual quality and anatomical accuracy of the generated image (i.e., synthetic brain MRI data). By using the latent feature representation or its concatenation with random noise as input, the controllable diversity of the generation results can be effectively increased. At the same time, each level uses a second 3D instance normalization layer, which can ensure that the individual-specific statistical information is maintained step by step during the reconstruction process, preventing the generation of homogeneous averaged images. This further solves the problem that traditional normalization methods (such as BatchNorm) will average individual features during batch processing, resulting in generated images that tend to be templated and cannot reflect the anatomical variability of different subjects, thus causing the loss of individual differences during the generation process.

[0141] To further address the challenge of traditional spatial convolution in modeling global dependencies in brain structures—namely, the complex spatial correlations and long-range dependencies of brain structures, and the difficulty of effectively capturing global connections between key structures such as sulci and white matter boundaries due to the limitations of local receptive fields—this application provides a scalp-to-brain magnetic resonance imaging generative model training method. The neural network module 50 based on a frequency domain attention mechanism includes a three-dimensional frequency domain self-attention module (FDSA3D) 51. The FDSA3D module 51 performs self-attention calculation in the frequency domain using Fast Fourier Transform.

[0142] It is understandable that the aforementioned frequency domain sharpening module 80 can also be implemented using a three-dimensional frequency domain self-attention module 51. When the three-dimensional frequency domain self-attention module 51 is used as a specific implementation of the neural network module 50 based on the frequency domain attention mechanism, it is embedded in any level of upsampling block. When the three-dimensional frequency domain self-attention module 51 is used as a specific implementation of the frequency domain sharpening module 80, it is placed at the end of the generator.

[0143] In one or more embodiments of this application, the three-dimensional frequency domain self-attention module can be abbreviated as FDSA3D or written as 3D frequency domain self-attention module.

[0144] Among them, see Figure 5 The three-dimensional frequency domain self-attention module 51 specifically includes the following:

[0145] (1) The third three-dimensional instance normalization layer 511 is used to perform feature-by-feature three-dimensional instance normalization processing on the third input feature and output the third normalized feature; wherein, when the three-dimensional frequency domain self-attention module 51 is integrated into the encoder 10, the third input feature is the first normalized feature output by the previous downsampling coding block 60; when the three-dimensional frequency domain self-attention module 51 is integrated into the generator 20, the third input feature is the second activated feature output by the previous upsampling generation block 70.

[0146] (2) A parallel branch processing unit 512, connected to the third 3D instance normalization layer 511, includes: a first branch 5121 for processing the third normalized features to generate a query matrix, a second branch 5122 for processing the third normalized features to generate a key matrix, and a third branch 5123 for processing the third normalized features to generate a value matrix. Specifically, the parallel branch processing unit 512 refers to a network structure design in which three branches process the same input (the third normalized feature) simultaneously and independently. Each branch typically includes operations such as convolution, which are used to generate the query matrix, key matrix, and value matrix in the attention mechanism, respectively. The query matrix and key matrix are used to calculate the similarity between features (attention weights), while the value matrix stores the feature information that needs to be aggregated according to the weights.

[0147] (3) The first fast Fourier transform unit 513 is connected to the parallel branch processing unit 512 and is used to transform the query matrix and the key matrix from the spatial domain to the frequency domain.

[0148] (4) A frequency domain attention calculation unit 514, connected to the first fast Fourier transform unit 513, is used to calculate attention weights in the frequency domain based on the query matrix and the key matrix, and to use the attention weights to perform weighted fusion on the value matrix, outputting frequency domain weighted features; specifically, in the frequency domain, the frequency domain attention calculation unit 514 receives the frequency domain representations of the query and key from the fast Fourier transform, calculates the correlation between them to obtain attention weights, and then uses these weights to perform weighted fusion on the value matrix, which is also in the frequency domain. Its output is the frequency domain feature after attention weighting.

[0149] (5) The first inverse fast Fourier transform unit 515 is connected to the frequency domain attention calculation unit 514 and is used to convert the frequency domain weighted features from the frequency domain to the spatial domain and output the spatial domain features. Specifically, the first inverse fast Fourier transform unit 515 performs an inverse fast Fourier transform operation to convert the features after frequency domain attention weighted fusion from the frequency domain back to the spatial domain so as to perform subsequent spatial domain processing.

[0150] (6) Output adjustment layer 516, connected to the first inverse fast Fourier transform unit 515, is used to adjust the channels of the spatial domain features, output the first self-attention feature, and add the first self-attention feature and the third input feature through residual connection to form the first target frequency domain self-attention feature; wherein, when the three-dimensional frequency domain self-attention module 51 is integrated into the encoder 10, the first target frequency domain self-attention feature is the first frequency domain self-attention feature; when the three-dimensional frequency domain self-attention module 51 is integrated into the generator 20, the first target frequency domain self-attention feature is the second frequency domain self-attention feature. Specifically, the output adjustment layer 516 can refer to a 1×1×1 convolutional layer, used to adjust or integrate the channel dimension of the spatial domain features converted from the frequency domain, so that it matches the original input features in the channel dimension, preparing for residual connection. The residual connection addition means directly performing element-wise addition operation between the module output (here, the first self-attention feature) and the module's original input (the third input feature). This constitutes a short-circuit connection, which helps gradient flow and model learning identity mapping. Specifically, the first target frequency domain self-attention feature is a feature that has been enhanced by frequency domain self-attention and fused with the original input information. Whether it is specifically called the first frequency domain self-attention feature or the second frequency domain self-attention feature depends on whether this module is integrated into the encoder or the generator.

[0151] As described above, the scalp-to-brain magnetic resonance imaging generative model training method provided in this application transforms the query and key matrices to the frequency domain through a first fast Fourier transform unit, and performs element-level operations in the frequency domain (rather than the massive matrix multiplication in the spatial domain) to calculate the correlation, effectively reducing the complexity of attention calculation and making it feasible to apply global self-attention on three-dimensional high-resolution data, thus achieving a significant reduction in computational complexity. The frequency domain attention calculation unit completes the core weight calculation and feature weighting in the frequency domain, which can effectively model the long-range dependency relationship of features in the global frequency range. The input is standardized through a third three-dimensional instance normalization layer, providing a stable distribution for subsequent calculations. The residual connection addition operation after the output adjustment layer forms a residual learning block, which can ensure that the gradient can be effectively backpropagated, alleviate the possible gradient vanishing problem, and allow the module to focus on learning the residual enhancement part of the input features, making the deep network training more stable.

[0152] To further address the issues of missing high-frequency details and blurred textures in generated images—specifically, the loss of high-frequency information such as brain sulci and cortical textures due to excessive smoothing during the generation process in existing methods, which affects the diagnostic usability of the images—this application provides a scalp-to-brain magnetic resonance imaging (MRI) generation model training method. The neural network module 50 based on a frequency domain attention mechanism includes a three-dimensional frequency-domain feed-forward network module (Frequency-Domain Feed-Forward Network 3D, FDFFN3D) 52. This three-dimensional frequency-domain feed-forward network module can be abbreviated as FDFFN3D or simply written as a 3D frequency-domain feed-forward network module.

[0153] Among them, see Figure 6 The three-dimensional frequency domain feedforward network module 52 specifically includes the following components:

[0154] (1) The fourth three-dimensional instance normalization layer 521 is used to perform feature-by-feature three-dimensional instance normalization processing on the fourth input feature and output the fourth normalized feature; wherein, when the three-dimensional frequency domain feedforward network module 52 is integrated into the encoder 10, the fourth input feature is the first normalized feature output by the previous level downsampling coding block 60; when the three-dimensional frequency domain feedforward network module 52 is integrated into the generator 20, the fourth input feature is the second activated feature output by the previous level upsampling generation block 70.

[0155] (2) Channel transformation layer 522, connected to the fourth three-dimensional instance normalization layer 521, is used to transform the channel dimension of the fourth normalized feature and output the feature in the form of a three-dimensional tensor.

[0156] (3) The image block serialization unit 523, connected to the channel transformation layer 522, is used to unfold the features in the form of a three-dimensional tensor into a sequence of image blocks. Specifically, the image block serialization unit is an operation unit that divides a three-dimensional feature tensor (with dimensions [C, D, H, W], where C is the number of channels, and D, H, and W are the depth, height, and width, respectively) into multiple local three-dimensional blocks, flattens these blocks, and arranges them into a two-dimensional sequence ([N, P], where N is the number of blocks and P is the number of features in each flattened block). It is a key step in converting regular grid data into a sequence representation for subsequent transformations (such as FFT). In the image block serialization unit 523, this operation allows the subsequent Fast Fourier Transform (FFT) to be performed independently on each local block, realizing local frequency domain analysis of the three-dimensional data. This processing method is more computationally efficient than the global three-dimensional fast fourier transform (3D FFT) and can better capture the frequency characteristics of local regions.

[0157] (4) The second fast Fourier transform unit 524 is connected to the image block serialization unit 523 and is used to convert the image block sequence from the spatial domain to the frequency domain.

[0158] (5) A learnable frequency domain modulation unit 525, connected to the second fast Fourier transform unit 524, is used to multiply the transformed features element-wise with a learnable weight matrix in the frequency domain to output the modulated spectrum. Specifically, the learnable frequency domain modulation unit is a learnable complex weight matrix (or a real weight matrix, applied to the amplitude spectrum). This unit multiplies the feature spectrum (complex form) obtained by FFT with this complex weight matrix element-wise (Hadamard product) to achieve adaptive enhancement or attenuation of different frequency components. Driven by training data, this complex weight matrix can learn the frequency bands most important for the scalp-to-brain mapping task. For example, it can learn to enhance high-frequency components representing tissue boundaries and fine textures while weakening specific frequencies that may contain noise or unimportant information, thereby achieving targeted frequency domain feature enhancement.

[0159] (6) The second inverse fast Fourier transform unit 526 is connected to the learnable frequency domain modulation unit 525 and is used to convert the modulated spectrum from the frequency domain back to the spatial domain and output the spatial domain sequence.

[0160] (7) Image block recombination unit 527 is connected to the second inverse fast Fourier transform unit 526 and is used to recombine the spatial domain sequence into recombination features in the form of three-dimensional tensors.

[0161] (8) A gated activation layer 528, connected to the image patch reconstruction unit 527, is used to perform nonlinear activation on the reconstructed features in the form of a three-dimensional tensor, outputting a first feedforward activation feature, and adding the first feedforward activation feature to the fourth input feature through a residual connection to form a second target frequency domain feedforward feature; wherein, when the three-dimensional frequency domain feedforward network module 52 is integrated into the encoder 10, the second target frequency domain feedforward feature is the first frequency domain feedforward feature; when the three-dimensional frequency domain feedforward network module 52 is integrated into the generator 20, the second target frequency domain feedforward feature is the second frequency domain feedforward feature. Specifically, the gated activation layer is a nonlinear activation function layer that uses a gating mechanism, such as a gated exponential linear unit (GEGLU). It typically transforms the input linearly into two parts, one part using a sigmoid function as a gate to control the amount of information passing through the other part (usually after a linear transformation or identity transformation), and the two results are multiplied element-wise. This layer can apply a nonlinear transformation to the features after conversion and reconstruction from the frequency domain. Its gating mechanism can more finely regulate the information flow and enhance the model's expressive power. Combined with residual connections, it introduces necessary nonlinearity while ensuring a direct path for the original information, which is beneficial for optimization.

[0162] As described above, the scalp-to-brain magnetic resonance imaging (MRI) generation model training method provided in this application reduces computational burden through image patch serialization, frequency domain modulation, and reconstruction. It utilizes learnable frequency domain modulation units to adaptively weight the three-dimensional feature spectrum, enabling the model to autonomously enhance frequency components crucial to the generation task (such as high frequencies corresponding to anatomical details) and suppress irrelevant or noise frequencies. The fourth three-dimensional instance normalization layer performs sample-independent normalization of features at the entry point, reducing internal covariate shifts. The residual connection adds the gated output to the original input, effectively mitigating gradient vanishing, preserving low-frequency information, and significantly improving the training stability and convergence speed of deep networks. The frequency domain modulation capability of the three-dimensional frequency domain feedforward network module, combined with its multiple calls throughout the network (integrated into various levels of the encoder and generator), allows the model to perform refined modeling of frequency domain information at multiple feature levels. This directly improves the accuracy of the synthesized images in terms of spatial structure and the clarity of high-frequency anatomical details (such as brain sulci and cortical textures). This is reflected in quantitative indicators as an increase in the Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR) and a decrease in Learned Perceptual Image Patch Similarity (LPIPS).

[0163] To further address the problem of insufficient multi-scale structural information fusion, namely the need for brain image generation to simultaneously consider global structure (such as lobe distribution) and local details (such as sulcus morphology), and the difficulty of comprehensively measuring generation quality with a single-scale loss function, a scalp-to-brain magnetic resonance imaging generation model training method is provided in this application embodiment. In this method, the composite loss function is a weighted sum of multiple losses, and the weights are determined by validation set grid search.

[0164] These losses specifically include:

[0165] (1) The first adversarial loss term is calculated based on the image discriminator 40’s distinction between the synthetic brain magnetic resonance imaging data and the real brain magnetic resonance imaging sample, and is used to optimize the generator; wherein, the first adversarial loss term can also be called image adversarial loss.

[0166] (2) The second adversarial loss term is calculated based on the distinction result between the latent feature representation and the random noise vector by the feature discriminator 30 and is used to optimize the encoder; the second adversarial loss term can also be called feature adversarial loss.

[0167] The first adversarial loss term and the second adversarial loss term are both adversarial losses (discriminator losses). The first adversarial loss term comes from the image discriminator and corresponds to the loss in the model framework that distinguishes between real and synthetic images. The second adversarial loss term comes from the feature discriminator and corresponds to the loss that constrains the distribution of latent feature representations (against random noise). Together, they constitute a complete adversarial training objective.

[0168] (3) The frequency domain loss term is obtained by converting the synthetic brain magnetic resonance imaging data and the real brain magnetic resonance imaging sample to the frequency domain and calculating the difference in frequency domain representation; wherein, the frequency domain loss term can specifically adopt asymptotic frequency domain L1 (absolute error), and can be specifically set as follows: The window (i.e., window sizes of 16×16×16 voxels, 8×8×8 voxels, and 4×4×4 voxels) has a small window proportion that increases with the training epoch t according to the S-curve.

[0169] (4) The multi-scale spatial loss term is obtained by calculating the absolute error between the synthesized brain MRI data and the real brain MRI sample on image patches of different scales; wherein, the multi-scale spatial loss term can specifically adopt multi-scale sliding L1 (absolute error), and can be specifically set as follows: The window is used to calculate the total loss, which is a weighted sum of the losses at each scale. The weights can be set to 0.5, 0.3, and 0.2, respectively. This emphasizes calculating the pixel-level absolute error (L1 norm) between images at multiple scales (16³ / 8³ / 4³ windows) to constrain the global and local structures.

[0170] (5) The perceptual similarity loss term is obtained by comparing the difference between the depth feature representations of the synthetic brain magnetic resonance imaging data and the real brain magnetic resonance imaging samples. Specifically, the perceptual similarity loss term can adopt learnable perceptual loss (LPIPS), which uses two-dimensional slice multi-scale in the early stage and orthogonal three-view averaging in the later stage; it is one of the specific implementation methods of perceptual similarity loss based on depth feature representation.

[0171] Based on this, the composite loss function may also include a feature matching loss term, namely:

[0172] (6) Feature matching loss term, which is obtained by calculating the difference between the features extracted by the intermediate multilayer network layers of the encoder from the scalp MRI sample and the synthetic brain MRI data. Specifically, the feature matching loss term can use perceptual loss (absolute error of the features of the intermediate four layers of the encoder) as an additional, internal feature matching loss.

[0173] Specifically, the composite loss function is shown in Formula (I) below:

[0174] Formula (1)

[0175] In formula (1), This represents the generator composite loss (i.e., the total loss). This represents the first adversarial loss (i.e., image adversarial loss); This represents the weight of the first adversarial loss and can be set to 1; This represents the second adversarial loss (i.e., feature adversarial loss). This represents the multi-scale spatial loss (i.e., multi-scale sliding L1). The weight representing the multi-scale spatial loss can be set to 10; This represents frequency domain loss (i.e., asymptotic frequency domain loss). The weight representing the frequency domain loss can be set to 5; This represents the perceptual similarity loss (i.e., the learning perceptual loss). The weight representing the perceptual similarity loss can be set to 1; This represents the feature matching loss (i.e., perceptual loss). This represents the weight of the feature matching loss term and can be set to 0.5.

[0176] As described above, the scalp-to-brain magnetic resonance imaging (MRI) generative model training method provided in this application integrates five losses—image realism, feature distribution, frequency domain details, spatial structure, and semantic content—into an optimizable objective. Through (implicit) weight allocation, the model can simultaneously pursue optimal performance across multiple dimensions, rather than optimizing a single metric in isolation, thereby generating images that perform well across multiple evaluation criteria. Different loss terms calculate errors and generate gradients from different perspectives. The first and second adversarial losses provide adversarial gradients to drive pattern search; the frequency domain loss and multi-scale spatial loss provide strongly constrained regressive gradients to ensure structural accuracy; and the perceptual similarity loss provides semantic-based gradients to improve visual naturalness. The combination of these gradients makes the training process more stable, and the converged model has stronger generalization ability. The frequency domain loss is directly related to the recovery of high-frequency details (improving sharpness), the multi-scale spatial loss ensures structural fidelity from global contours to local grooves (improving SSIM), and the perceptual similarity loss ensures that the image is visually closer to the real image (reducing LPIPS). The combination of these effects makes the generated images more suitable for subsequent clinical analysis and diagnosis in terms of both visual quality and quantitative indicators.

[0177] To further ensure individual characteristics during training, in one embodiment of the scalp-to-brain magnetic resonance imaging generative model training method of this application, the feature discriminator includes multiple fully connected layers for processing the latent feature representation and random noise vectors sampled from the preset probability distribution and outputting a first scalar score. The first scalar score is used to construct a second adversarial loss term in the composite loss function to constrain the distribution of the latent feature representation to be close to the preset probability distribution. Each fully connected layer of the feature discriminator 30 is sequentially connected to a fifth three-dimensional instance normalization layer and a fifth activation function layer.

[0178] Furthermore, in order to further improve the adaptability of the image discriminator to three-dimensional medical image data, in an embodiment of the scalp-to-brain magnetic resonance imaging generative model training method of this application, the image discriminator is a discriminator based on a three-dimensional convolutional neural network, which is used to process the synthetic brain magnetic resonance imaging data and the real brain magnetic resonance imaging samples and output a second scalar score. The second scalar score is used to construct the first adversarial loss term in the composite loss function, so that the first adversarial loss term is calculated based on the second scalar score, and a gradient penalty is introduced in the calculation process to follow the Wasserstein distance optimization criterion.

[0179] Based on the aforementioned embodiments of the scalp-to-brain magnetic resonance imaging (MRI) generation model training method, this application also provides an embodiment of the scalp-to-brain MRI generation method, see [link to embodiment]. Figure 7 The method for generating scalp-to-brain magnetic resonance imaging specifically includes the following:

[0180] Step 300: Obtain scalp magnetic resonance imaging data of the target object.

[0181] It is understood that the scalp magnetic resonance imaging data obtained in step 300 can be the aforementioned raw scalp magnetic resonance imaging data, or it can be the scalp magnetic resonance imaging data processed by the aforementioned preprocessing method.

[0182] Step 400: Input the scalp magnetic resonance imaging data into the scalp-to-brain magnetic resonance imaging generation model so that the scalp-to-brain magnetic resonance imaging generation model outputs corresponding synthetic brain magnetic resonance imaging data, wherein the scalp-to-brain magnetic resonance imaging generation model is trained based on the scalp-to-brain magnetic resonance imaging generation model training method.

[0183] The embodiments of the scalp-to-brain magnetic resonance imaging generation method provided in this application mention the scalp-to-brain magnetic resonance imaging generation model training method. Specifically, it can be implemented using the processing flow of the scalp-to-brain magnetic resonance imaging generation model training method embodiments described above. Its functions will not be repeated here, but can be referred to the detailed description of the scalp-to-brain magnetic resonance imaging generation model training method embodiments described above.

[0184] To further illustrate the above embodiments, this application also provides an application example of a scalp-to-brain magnetic resonance imaging (MRI) model training method. This application example aims to propose a method for generating high-quality three-dimensional brain MRI images based on scalp MRI data. By introducing a frequency domain attention mechanism and an individual feature preservation strategy, it achieves accurate mapping from scalp structures to brain tissue images, solving problems such as lack of structural guidance, insufficient detail modeling, and loss of individual differences in existing brain image generation technologies.

[0185] The overall workflow provided in this application example mainly includes three stages: scalp feature extraction and encoding, image generation, and multi-domain joint supervised training. See also... Figure 8 These stages are completed collaboratively through a generative network framework consisting of an encoder, generator, image discriminator, and feature discriminator. Details are as follows:

[0186] Phase 1: Scalp Feature Extraction and Encoding Phase

[0187] This stage is primarily performed by the encoder. First, the input scalp MRI data is preprocessed, including normalizing voxel intensities to the [-1, 1] interval and uniformly cropping them to a preset size (e.g., 64×64×64 voxels). Then, the preprocessed data is input to the encoder. This encoder can be, for example, a three-dimensional convolutional neural network with multi-level downsampling, incorporating frequency-domain attention-based neural network modules (e.g., frequency-domain network modules, FDNetBlock) in some network layers to enhance feature extraction capabilities. The encoder ultimately outputs a low-dimensional latent feature representation, i.e. Figure 8 The encoded features in the model. Meanwhile, the feature discriminator introduced in this application example processes the latent feature representation and a random noise, outputting a first scalar score (i.e., Figure 8 The latent Wasserstein distance is used to construct a second adversarial loss term during training to constrain the distribution of latent feature representations.

[0188] Second stage: Image generation stage:

[0189] This stage is completed by the generator. The generator performs image synthesis using the latent feature representation obtained in the first stage and random noise as conditions. The generator can be, for example, a three-dimensional decoding network containing multi-level upsampling, with the frequency domain network module (FDNetBlock) also embedded in some network levels. Specifically, a frequency domain self-attention submodule (FDSA3D) can be introduced at the end of the generator to further sharpen the high-frequency anatomical details of the generated image. The generator finally outputs the synthesized brain magnetic resonance imaging data (i.e., ...) through a convolutional layer and an activation function (such as the Tanh function). Figure 8 (Image generated in the image).

[0190] Then, the generated image and the real brain MRI sample are input into the image discriminator. The image discriminator processes the real and synthetic brain MRI data and outputs a second scalar score, i.e. Figure 8 The Wasserstein distance in the image is calculated and a gradient penalty is introduced in the calculation, following the Wasserstein distance optimization criterion.

[0191] Phase 3: Multi-domain Joint Supervision Training Phase

[0192] This stage uses a composite loss function, as shown in Formula (I) above, to drive the training of the entire generative network framework.

[0193] It should be noted that the frequency domain network module (FDNetBlock) internally includes repeated frequency domain self-attention submodules (FDSA3D) and / or frequency domain feedforward network submodules (FDFFN3D). Specifically: the FDSA3D submodule models global context information by calculating self-attention weights in the frequency domain. The FDFFN3D submodule adaptively modulates different frequency components by element-wise multiplying features with a learnable weight matrix in the frequency domain, and finally outputs the data through activation functions such as Gated Exponential Linear Units (GEGLU) and residual connections.

[0194] Specifically, see Figure 9 The 3D frequency domain self-attention module effectively captures global correlations between brain regions by transforming images to the frequency domain and using element-wise multiplication instead of traditional spatial domain matrix multiplication, while reducing computational complexity. The specific workflow of the 3D frequency domain self-attention module (FDSA3D) is as follows:

[0195] (1) Feature Preprocessing and Branch Generation: The input features are first processed by instance normalization to eliminate batch dependencies and enhance model stability. Subsequently, the features are fed into three parallel processing branches. Each branch contains a 1×1×1 convolutional layer (for channel transformation) and a 3×3×3 convolutional layer (for extracting local spatial context information). The outputs of these three branches serve as the inputs to the query matrix, key matrix, and value matrix, respectively.

[0196] (2) Generation of frequency domain attention weights: Before entering the Fast Fourier Transform (FFT), the query matrix and the key matrix undergo dimension matching to ensure consistency in subsequent calculations. Subsequently, they are mapped to the frequency domain via FFT. In the frequency domain, the dot product similarity between the query matrix and the key matrix is ​​calculated to obtain the global self-attention weights, thus realizing the modeling of global frequency correlation.

[0197] (3) Frequency domain feature modulation and spatial domain reconstruction: In the frequency domain, the self-attention weights (i.e., dot product similarity results) calculated in the previous step are weighted and fused with the value matrix (which is also transformed to the frequency domain by FFT). The fused features are then restored to the spatial domain by inverse fast Fourier transform (IFFT).

[0198] (4) Output generation: The features transformed back to the spatial domain are normalized again to stabilize the distribution, and then channel adjustment is performed through a 1×1×1 convolutional layer. Finally, the processing result is concatenated (added) with the original input features through residuals to form the final output of the module.

[0199] In other words, the 3D frequency domain self-attention module effectively captures long-range dependencies and frequency structures in 3D medical image data by introducing a self-attention mechanism in the frequency domain. At the same time, it maintains spatial locality by preserving spatial convolution branches, making it suitable for high-precision scalp-to-brain medical image generation tasks.

[0200] Specifically, see Figure 10 The 3D frequency domain feedforward network module enhances the ability to model high-frequency details, making it particularly suitable for feature extraction of key structures such as brain sulci and white matter boundaries. The specific workflow of the 3D frequency domain feedforward network module (FDFFN3D) is as follows:

[0201] (1) Instance normalization and channel transformation (corresponding to the fourth three-dimensional instance normalization layer 521 and channel transformation layer 522): The input features are first processed by instance normalization to eliminate batch dependency and stabilize the training process. Then, a 1×1×1 three-dimensional convolutional layer is used to transform and compress the channel dimension, and the features are output in the form of three-dimensional tensors.

[0202] (2) Image patch serialization and complex frequency domain conversion (corresponding to image patch serialization unit 523 and second fast Fourier transform unit 524): The features in the form of three-dimensional tensors are converted into an image patch sequence through the image patch unfolding operation. This sequence is mapped to a complex frequency domain representation via fast Fourier transform (FFT) to obtain global frequency information.

[0203] (3) Learnable frequency domain adaptive modulation (corresponding to learnable frequency domain modulation unit 525): In the frequency domain, the transformed features are multiplied element by element by a learnable weight matrix to achieve adaptive modulation of different frequency components—for example, enhancing high-frequency details or suppressing noise bands.

[0204] (4) Spatial domain reconstruction and reassembly (corresponding to the second inverse fast Fourier transform unit 526 and the image patch reassembly unit 527): The modulated spectrum is reconstructed back to the spatial domain through inverse fast Fourier transform (IFFT) and restored to the original three-dimensional tensor shape through image patch folding.

[0205] (5) Gated nonlinear activation and residual fusion (corresponding to gated activation layer 528 and residual connections): The recombined 3D features are fed into the GEGLU (Gated Exponential Linear Unit) activation function, which introduces strong nonlinear expressive power through a gating mechanism. Finally, the activation result is fused with the original input features through residual connections (element-wise addition) to form the final output of the module.

[0206] In other words, the entire process of the 3D frequency domain feedforward network module is trained end-to-end within a differentiable framework, focusing on jointly modeling local channel transformations and global frequency domain structure. By introducing learnable frequency domain modulation and gated activation mechanisms, it effectively enhances the ability to model high-frequency anatomical details. Its input-output same-dimensional residual connection design not only effectively alleviates the gradient vanishing problem and preserves low-level feature information, but also enables the module to be plug-and-play integrated as a standardized component into various levels of the encoder or generator network, making it one of the key innovative modules for generating high-quality, high-fidelity brain MRI images.

[0207] In other words, the method provided in the application examples of this application has the following beneficial effects:

[0208] (1) The application example of this application establishes the mapping relationship between the scalp and brain structures by constructing a generative model based on scalp MRI, thereby improving the biological rationality and clinical applicability of the generated images.

[0209] (2) The application example of this application introduces the frequency domain self-attention mechanism (FDSA3D) to realize global feature modeling in the frequency domain, which significantly improves the structural integrity and topological fidelity of the generated image.

[0210] (3) The application example of this application designs a frequency domain feedforward network module (FDFFN3D) to enhance the modeling ability of high frequency components, effectively restore the fine structure in brain images, and improve the image clarity and anatomical accuracy.

[0211] (4) The application example of this application uses three-dimensional instance normalization (InstanceNorm3D) to normalize each sample, retain the structural features of each individual, enhance the personalized expression ability of the generated image, and meet the needs of clinical individualized analysis.

[0212] (5) In the application example, this application proposes a composite loss function that combines multi-scale sliding window L1 loss, progressive frequency domain loss and LPIPS perceptual loss to achieve synchronous optimization of multi-level structural information and improve the overall consistency and local realism of the generated image.

[0213] (6) The application example of this application constructs a complete model framework, which can integrate frequency domain attention module, individual normalization strategy and multi-scale loss mechanism, and can form a generalizable and scalp-driven brain image generation system, providing a new technical path for neuroimaging analysis, disease auxiliary diagnosis and brain structure modeling.

[0214] To further illustrate the above embodiments and application examples, this application also provides an execution example of the scalp-to-brain magnetic resonance imaging (MRI) generation model training method and the scalp-to-brain MRI generation method, which specifically includes:

[0215] (1) Operating environment configuration:

[0216] Hardware environment: A computer equipped with an NVIDIA graphics processing unit (GPU) with a video memory capacity of not less than 12 GB;

[0217] Software environment: The operating system can be Ubuntu 20.04 or Windows 10; the programming language is Python 3.10; the deep learning framework is PyTorch (version no lower than 2.2.1).

[0218] Dependencies: The following Python packages need to be installed: PyTorch deep learning library, NiBabel medical image reading and writing library, SciPy scientific computing library, NumPy numerical computing library, Matplotlib plotting library, TensorBoard visualization toolkit, Learned Perceptual Image PatchSimilarity (lpips) toolkit, and TorchIO medical image processing toolkit.

[0219] (2) Data preparation process: The data used for model training can be preprocessed according to the following steps:

[0220] Cortical reconstruction was performed on the raw T1-weighted MRI data using the full-process reconstruction command of the FreeSurfer 7.3 brain surface analyzer software;

[0221] The SynthStrip deep learning tool was used to separate the scalp from the brain.

[0222] All image data were uniformly resampled to a 1 mm isotropic resolution;

[0223] The images were registered to the Montreal Neuroscience Institute 152 standard brain template with 1 mm isotropic resolution using affine transformation, i.e., MNI152 1 mm standard brain template.

[0224] The image is cropped or padded to a fixed size of 64×64×64 voxels, and its voxel intensity values ​​are linearly mapped to the interval [-1, 1].

[0225] Divide the dataset into training and testing sets. For example, you can use the publicly available database from the Human Connectome Project (HCP), using 1000 data points as the training set and 112 data points as the testing set.

[0226] (3) Model training steps: You can download and use the pre-trained model parameters, or you can train from scratch based on the training script (such as running "python train.py" in the command line).

[0227] (4) Image generation (reasoning):

[0228] After training, the parameters of the encoder and generator are saved. In application, new scalp MRI data is input into the encoder, and the obtained latent feature representations are input into the generator to obtain the corresponding synthetic brain MRI image.

[0229] To further illustrate the beneficial effects of the methods described in the embodiments and application examples of this application, this application also provides corresponding experimental analysis. Specifically, it employs the deep learning framework PyTorch to conduct generation quality assessment and volume correlation analysis on three-dimensional medical images. The input data is organized in the form of a five-dimensional tensor with the shape [N,C,D,H,W], where N represents the batch size, C represents the number of channels, D represents the depth, H represents the height, and W represents the width.

[0230] For perceptual quality evaluation, a pre-trained perceptual distance network (LPIPS) is used as the core metric. In the specific implementation, the loss function interface (loss_fn_vgg) provided by LPIPS is used to slice the 3D volume data into 2D images along the axis (or other specified direction), and the perceptual similarity is calculated for each slice. Finally, the average is taken as the overall perceptual loss.

[0231] Statistical characteristic analysis relies on the numerical computing library (numpy) and the statistical computing library (scipy.stats), covering multiple dimensions such as mean, standard deviation, skewness, kurtosis, KL divergence (Kullback–Leibler Divergence), and Wasserstein distance (also known as bulldozer distance, or EarthMover's Distance, EMD), to quantify the differences in intensity distribution and morphological features between real and generated images.

[0232] The visualization module is based on the matplotlib plotting library and includes: volume correlation scatter plots (showing the comparison between the actual volume and the generated volume); intensity / volume distribution histograms and kernel density estimation curves; difference heatmaps (such as voxel error plots or Jacobian deformation field visualizations), etc., to intuitively reveal the spatial patterns of generation bias.

[0233] All experimental metrics and intermediate results are written to structured log files through a unified log interface (log_losses), supporting systematic comparisons and backtracking analysis across models and rounds.

[0234] Core process description:

[0235] 1. Input data preparation: Obtain paired 3D medical images and their corresponding skull images as conditional inputs;

[0236] 2. Model inference: Input the difference map into encoder E to obtain the latent vector z, and then reconstruct the T1 image by generator G;

[0237] 3. Multi-dimensional quality assessment:

[0238] (1) Calculate the absolute error loss at the voxel level;

[0239] (2) Slice along the three spatial dimensions and calculate the average learnable perceptual loss;

[0240] (3) Calculate the structural similarity index and the multidimensional structural similarity index using a 3D convolutional window;

[0241] (4) Calculate the peak signal-to-noise ratio based on the dynamic range;

[0242] 4. Anatomical structure fidelity analysis:

[0243] (1) Set a value (e.g., -0.5) to binarize the image and count the number of non-zero voxels as the "brain volume";

[0244] (2) Calculate the Pearson correlation coefficient between the actual volume and the generated volume of the 112 samples;

[0245] (3) Draw scatter plots and distribution histograms to verify the consistency of volume distribution;

[0246] 5. Pixel-level distribution analysis: Compare the pixel value distribution of the real and generated images;

[0247] 6. Results visualization and log recording: Automatically saves image comparison charts, difference charts, and statistical charts, and writes the indicators to the log.

[0248] in, Figure 11 The paper presents comparative results of the generation of central slices from real brain magnetic resonance imaging data with those of a 3D variational autoencoder, the model of this application with the introduction of a frequency domain network module (FDNetBlock) (i.e., a Wasserstein generative adversarial network that integrates a frequency domain attention mechanism and has gradient penalty), a 3D encoder generative adversarial network (3D-Encoder-WGAN-GP), a 3D α adversarial generative network (3D-αWGAN-GP), the model of this application using batch normalization (BatchNorm), and the model of this application using instance normalization (InstanceNorm). The first column shows real brain MRI center slices. The remaining columns, in order, show the slices generated by the following models: a 3D variational autoencoder, the model incorporating a frequency domain network module (FDNetBlock) (i.e., a Wasserstein generative adversarial network with a gradient penalty and incorporating frequency domain attention mechanism), a 3D encoder-WGAN-GP, a 3D α-adversarial generative network (3D-αWGAN-GP), the model in this application using batch normalization (BatchNorm), and the model in this application using instance normalization (InstanceNorm). The last column shows the input (i.e., scalp MRI data or samples). Figure 11 As can be clearly seen, the images generated by the model proposed in this application significantly outperform the baseline model in terms of detail fidelity and volume correlation, exhibiting clearer textures and more accurate anatomical structures. In particular, when the frequency domain network module is removed or 3D batch normalization (BatchNorm3D) is used instead of 3D instance normalization (InstanceNorm3D), the image details become noticeably blurred and the volume error increases.

[0249] Figures 12(a) to 12(f) respectively show the numerical distribution comparison of synthetic brain MRI data (hereinafter referred to as synthetic in Figures 12(a) to 12(f)) and real brain MRI data (hereinafter referred to as real in Figures 12(a) to 12(f)) for the following models: 3D variational autoencoder, the model of this application with the introduction of a frequency domain network module (FDNetBlock) (i.e., a Wasserstein generative adversarial network with a fused frequency domain attention mechanism and gradient penalty), 3D encoder generative adversarial network (3D-Encoder-WGAN-GP), 3D α adversarial generative network (3D-αWGAN-GP), the model of this application using batch normalization (BatchNorm), and the model of this application using instance normalization (InstanceNorm). All models generate data that exhibit a similar normal distribution in the range of values ​​greater than -0.4, while the original data distribution is multimodal. This finding raises the question of whether a multimodal distribution should be adopted for feature encoding to obtain better performance, which warrants further investigation.

[0250] Furthermore, Figures 13(a) to 13(f) illustrate the relationship between the brain volume generated by different models on the test set and the actual brain volume. Wherein:

[0251] Figure 13(a) illustrates the relationship between synthetic brain MRI data (hereinafter referred to as generative in Figures 13(a) to 13(f)) generated by the 3D variational autoencoder on the test set and real brain MRI data (hereinafter referred to as real data in Figures 13(a) to 13(f)).

[0252] Figure 13 (b) shows the relationship between synthetic brain MRI data and real brain MRI data generated on the test set by the model of this application with the introduction of frequency domain network modules (i.e., a frequency domain attention mechanism fused with a gradient-penalized Wasserstein generative adversarial network).

[0253] Figure 13(c) shows the relationship between synthetic brain MRI data generated by the 3D encoder generative adversarial network on the test set and real brain MRI data.

[0254] Figure 13(d) illustrates the relationship between synthetic brain MRI data generated on the test set and real brain MRI data by the 3D-αWGAN-GP model encoder generative adversarial network.

[0255] Figure 13(e) illustrates the relationship between synthetic brain MRI data generated on the test set using the batch-normalized model of this application and real brain MRI data.

[0256] Figure 13(f) illustrates the relationship between synthetic brain MRI data generated on the test set using the application model with instance normalization and real brain MRI data.

[0257] It can be seen that the 3D encoder-WGAN-GP generates a very narrow range of brain volumes due to the lack of encoder guidance. The 3D α-adversarial production network significantly expands the range of volume variations by imposing certain constraints on the encoder, but its correlation with the real brain volume is relatively low. The model in this application, which introduces a frequency domain network module (FDNetBlock), further expands the range of generated volumes, while the model in this application, which employs instance normalization, not only further expands the volume range but also significantly improves the correlation between the generated brain volume and the real brain volume.

[0258] In addition, the performance metrics comparison results of the above models on the test set are shown in Table 1.

[0259] Table 1: Performance metrics comparison results of the above models on the test set

[0260]

[0261] Table 1 shows the average performance of different models on five key image quality assessment metrics, including mean absolute error, structural similarity, multi-scale structural similarity, exponential peak signal-to-noise ratio, and perceptual similarity (LPIPS). Numerical results show that the 3D variational autoencoder (3D-VAE) performs best on most metrics, generating images that are most structurally similar to real images. The generative adversarial network (GAN)-based model has an advantage in detail fidelity. Notably, the 3D encoder GAN integrating a frequency domain network module and the model in this application, which incorporates a frequency domain network module, exhibit very similar performance across all metrics. The model in this application using batch normalization is an improvement on the 3D α-adversarial generative network; by replacing some network layers with frequency domain network modules, it achieves significant improvements across all metrics. Furthermore, the model in this application using instance normalization further improves the performance of multiple metrics by replacing batch normalization with instance normalization. These experimental results demonstrate that the frequency domain network module and instance normalization play a crucial role in enhancing the model's image generation capabilities.

[0262] In summary, the scalp-to-brain magnetic resonance imaging (MRI) model training method and the scalp-to-brain MRI generation method proposed in the embodiments of this application have the following beneficial effects:

[0263] (1) Pioneering a generative paradigm from scalp to brain:

[0264] For the first time, scalp MRI was used as the sole input condition, and the skull-brain morphological correlation was explicitly utilized, breaking through the limitations of traditional noise-to-brain or complete image-to-brain methods, and providing a new approach to brain structure inference for scenarios lacking MRI.

[0265] (2) Dual-domain collaborative modeling of frequency and spatial domains:

[0266] By embedding a frequency domain network module (FDNetBlock), and optionally FDSA3D and FDFFN3D, global self-attention computation is performed in the frequency domain, reducing the complexity from O(N²) to O(NlogN), while preserving the local detail capabilities of spatial convolution, which is significantly better than pure spatial convolution or traditional Transformer model solutions.

[0267] (3) High-frequency anatomical detail sharpening:

[0268] By introducing a 3D frequency domain self-attention module at the generation end, the spectrum of the three-dimensional fast Fourier transform (3DFFT) is directly weighted and then the inverse fast Fourier transform (IFFT) is performed back to the spatial domain, which improves the clarity of high-frequency structures such as cortical sulci and white matter boundaries by ≥18% and reduces the value of learning-aware image patch similarity (LPIPS) by 0.047.

[0269] (4) Preservation of individual characteristics:

[0270] The entire network uses 3D instance normalization (InstanceNorm3D) to normalize samples one by one, avoiding the cross-sample averaging effect of batch normalization. The correlation coefficient between the generated brain volume distribution and the real data is improved from 0.64 to 0.83, meeting the needs of personalized clinical analysis.

[0271] (5) Multi-scale and multi-domain composite loss:

[0272] We propose a triple constraint of progressive frequency domain loss, multi-window L1, and orthogonal LPIPS. The weights are automatically adjusted during training, which improves the structural similarity index by 4.7%, the peak signal-to-noise ratio by 1.8dB, and keeps the GPU memory usage <22GB (batch size=4).

[0273] (6) Plug-and-play modular design:

[0274] Both the frequency domain network module and FDSA3D can be independently embedded into any 3D-U-Net or generative adversarial network without modifying the backbone code. U-Net is a classic convolutional neural network architecture originally designed for biomedical image segmentation, named for its resemblance to the letter "U". 3D U-Net is an extended version for three-dimensional volumetric data (such as MRI and CT), using three-dimensional convolution for feature extraction.

[0275] (7) Broad application prospects:

[0276] It can be used for MRI data enhancement or missing modal completion of brain diseases, AI-assisted diagnosis, and promote the standardization and industrialization of brain connectomics and precision medicine research.

[0277] From a software perspective, this application also provides a scalp-to-brain magnetic resonance imaging (MRI) model training device for executing all or part of the scalp-to-brain MRI model training method. The embodiments of the scalp-to-brain MRI model training device provided in this application can specifically be used to execute the processing flow of the embodiments of the scalp-to-brain MRI model training method described above. Its functions will not be elaborated here, but can be referred to the detailed description of the embodiments of the scalp-to-brain MRI model training method described above.

[0278] The scalp-to-brain magnetic resonance imaging (MRI) generative model training device can perform the scalp-to-brain MRI generative model training portion in either a server or a client device. The choice can be made based on the processing power of the client device and limitations of the user's usage scenario. This application does not impose any limitations in this regard. If all operations are performed in the client device, the client device may further include a processor for specific processing of the scalp-to-brain MRI generative model training.

[0279] In addition, this application also provides a scalp-to-brain magnetic resonance imaging (MRI) generation device for performing all or part of the scalp-to-brain MRI generation method. The embodiments of the scalp-to-brain MRI generation device provided in this application can be used to perform the processing flow of the embodiments of the scalp-to-brain MRI generation method in the above embodiments. Its functions will not be repeated here, but can be referred to the detailed description of the embodiments of the scalp-to-brain MRI generation method described above.

[0280] The scalp-to-brain magnetic resonance imaging (MRI) generation device can perform the scalp-to-brain MRI generation process in either a server or a client device. The choice can be made based on the processing power of the client device and limitations imposed by the user's usage scenario. This application does not impose any limitations in this regard. If all operations are performed in the client device, the client device may further include a processor for the specific processing of the scalp-to-brain MRI generation.

[0281] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0282] The server and the client device can communicate using any suitable network protocol, including those not yet developed as of the date of this application. Such network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Furthermore, such network protocols may also include RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer Protocol) protocols used on top of the aforementioned protocols.

[0283] This application also provides an electronic device, which may include a processor, a memory, a receiver, and a transmitter. The processor is used to execute the scalp-to-brain magnetic resonance imaging (MRI) generative model training method and / or the scalp-to-brain MRI generative method mentioned in the above embodiments. The processor and the memory can be connected via a bus or other means, taking a bus connection as an example. The receiver can be connected to the processor and the memory via wired or wireless means.

[0284] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0285] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the scalp-to-brain magnetic resonance imaging (MRI) generative model training method and / or scalp-to-brain MRI generative method in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the scalp-to-brain MRI generative model training method and / or scalp-to-brain MRI generative method in the above method embodiments.

[0286] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0287] The one or more modules are stored in the memory, and when executed by the processor, the scalp-to-brain magnetic resonance imaging generation model training method and / or scalp-to-brain magnetic resonance imaging generation method in the implementation embodiment are executed.

[0288] In some embodiments of this application, the user equipment may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected via a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0289] As one implementation method, the functions of the receiver and transmitter in this application can be implemented by transceiver circuits or dedicated transceiver chips, and the processor can be implemented by dedicated processing chips, processing circuits or general-purpose chips.

[0290] As another implementation approach, the server provided in this application embodiment can be implemented using a general-purpose computer. That is, the program code implementing the processor, receiver, and transmitter functions is stored in memory, and the general-purpose processor implements the processor, receiver, and transmitter functions by executing the code in memory.

[0291] This application also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of the aforementioned scalp-to-brain magnetic resonance imaging (MRI) generation model training method and / or scalp-to-brain MRI generation method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0292] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned scalp-to-brain magnetic resonance imaging (MRI) generation model training method and / or scalp-to-brain MRI generation method.

[0293] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave.

[0294] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0295] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0296] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to the embodiments of this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for training a scalp-to-brain magnetic resonance imaging generative model, characterized in that, include: A frequency-domain attention-integrated, gradient-penalized Wasserstein generative adversarial network (GAN) is trained using a training dataset. The training dataset includes multiple sample pairs, each containing a scalp MRI sample and a corresponding real brain MRI sample. The frequency-domain attention-integrated, gradient-penalized Wasserstein GAN comprises: an encoder for receiving the scalp MRI sample and outputting a corresponding latent feature representation; a generator for generating corresponding synthetic brain MRI data based at least on the latent feature representation; and a generator for distinguishing the latent feature representation from random data sampled from a preset probability distribution. The system includes a feature discriminator for machine noise vectors and an image discriminator for distinguishing the synthetic brain MRI data from the corresponding real brain MRI samples; the discrimination result of the image discriminator is used to construct a first adversarial loss term, which is used to optimize the generator; the discrimination result of the feature discriminator is used to construct a second adversarial loss term, which is used to optimize the encoder; wherein the generator and / or the encoder integrate a neural network module based on a frequency domain attention mechanism; the encoder and the generator are configured to jointly constitute a scalp-to-brain MRI generative model after training; During the training of the fused frequency domain attention mechanism and gradient-penalized Wasserstein generative adversarial network, the encoder, generator, image discriminator and feature discriminator are jointly optimized using a composite loss function that includes a frequency domain loss term. The encoder includes multiple sequentially connected downsampling coding blocks for feature extraction and compression of the scalp magnetic resonance imaging sample, wherein each downsampling coding block includes: A first three-dimensional convolutional layer is used to perform convolution operations on the first input features and output downsampled features; wherein, the first input feature processed by the first three-dimensional convolutional layer in the first-level downsampled coding block is the scalp magnetic resonance imaging sample. The first activation function layer, connected to the first three-dimensional convolutional layer, is used to perform non-linear activation on the downsampled features and output the first activated features; The first 3D instance normalization layer is connected to the first activation function layer and is used to perform sample-by-sample 3D instance normalization processing on the first activated features and output the first normalized features. In the downsampling coding block at least one level, after the first three-dimensional instance normalization layer, a neural network module based on a frequency domain attention mechanism is integrated to process the first normalized features to output the first frequency domain self-attention features. When the neural network module based on the frequency domain attention mechanism is integrated into the last-level downsampling coding block, the first frequency domain self-attention feature is used as the latent feature representation, and the first normalized feature output by the downsampling coding block that is not the last level is used as the first input feature of the next-level downsampling coding block. When the neural network module based on the frequency domain attention mechanism is integrated into a downsampling coding block that is not the last level, the first frequency domain self-attention feature is the first input feature of the next level downsampling coding block, and the first normalized feature output by the last level downsampling coding block is the latent feature representation. The generator comprises multiple sequentially connected upsampling generation blocks for upsampling and reconstructing the latent feature representation, wherein each upsampling generation block includes: A deconvolutional layer is used to upsample the second input feature and output the upsampled feature; wherein, the second input feature processed by the deconvolutional layer in the first-level upsampling generation block is the latent feature representation, or is a concatenation feature of the latent feature representation and random noise; The second three-dimensional convolutional layer is connected to the deconvolutional layer and is used to perform convolution operations on the upsampled features to output convolutional features; The second 3D instance normalization layer is connected to the second 3D convolutional layer and is used to perform feature-by-feature 3D instance normalization processing on the convolutional features and output the second normalized features. The second activation function layer, connected to the second 3D instance normalization layer, is used to perform non-linear activation on the second normalized feature and output the second activated feature. In the at least one level of the upsampling generation block, after the second activation function layer, a neural network module based on a frequency domain attention mechanism is integrated to process the second activated features to output the second frequency domain self-attention features. The generator also includes: The frequency domain sharpening module, connected after the last-stage upsampling generation block, is used to perform frequency domain self-attention enhancement processing on the received features and output sharpened features. An output mapping layer, connected to the frequency domain sharpening module, is used to perform channel mapping and value domain transformation on the sharpening features and output the synthesized brain magnetic resonance imaging data. Wherein, when the neural network module based on the frequency domain attention mechanism is integrated into the last-level upsampling generation block, the neural network module based on the frequency domain attention mechanism passes the second frequency domain self-attention feature to the frequency domain sharpening module, and the second activated feature output by the upsampling generation block that is not the last level is used as the second input feature of the next level upsampling generation block; When the neural network module based on the frequency domain attention mechanism is integrated into the upsampling generation block that is not the last level, the second frequency domain self-attention feature is used as the second input feature of the next level upsampling generation block, and the second activated feature output by the last level upsampling generation block is passed to the frequency domain sharpening module.

2. The method for training a scalp-to-brain magnetic resonance imaging generative model according to claim 1, characterized in that, The neural network module based on the frequency domain attention mechanism includes a three-dimensional frequency domain self-attention module, which includes: The third 3D instance normalization layer is used to perform feature-by-feature 3D instance normalization processing on the third input feature and output the third normalized feature; wherein, if the 3D frequency domain self-attention module is integrated into the encoder, the third input feature is the first normalized feature output by the previous level downsampling coding block; when the 3D frequency domain self-attention module is integrated into the generator, the third input feature is the second activated feature output by the previous level upsampling generation block; A parallel branch processing unit is connected to the third 3D instance normalization layer. The parallel branch processing unit includes: a first branch for processing the third normalization feature to generate a query matrix, a second branch for processing the third normalization feature to generate a key matrix, and a third branch for processing the third normalization feature to generate a value matrix. The first Fast Fourier Transform unit, connected to the parallel branch processing unit, is used to transform the query matrix and the key matrix from the spatial domain to the frequency domain; A frequency domain attention calculation unit, connected to the first fast Fourier transform unit, is used to calculate attention weights in the frequency domain based on the query matrix and the key matrix, and to use the attention weights to perform weighted fusion on the value matrix to output frequency domain weighted features. The first inverse fast Fourier transform unit is connected to the frequency domain attention calculation unit and is used to transform the frequency domain weighted features from the frequency domain to the spatial domain and output the spatial domain features. An output adjustment layer, connected to the first inverse fast Fourier transform unit, is used to adjust the channels of the spatial domain features, output a first self-attention feature, and add the first self-attention feature and the third input feature through a residual connection to form a first target frequency domain self-attention feature; wherein, when the three-dimensional frequency domain self-attention module is integrated into the encoder, the first target frequency domain self-attention feature is the first frequency domain self-attention feature; when the three-dimensional frequency domain self-attention module is integrated into the generator, the first target frequency domain self-attention feature is the second frequency domain self-attention feature.

3. The method for training a scalp-to-brain magnetic resonance imaging generative model according to claim 1, characterized in that, The neural network module based on the frequency domain attention mechanism includes a three-dimensional frequency domain feedforward network module, which includes: The fourth 3D instance normalization layer is used to perform feature-by-feature 3D instance normalization processing on the fourth input feature and output the fourth normalized feature; wherein, when the 3D frequency domain feedforward network module is integrated into the encoder, the fourth input feature is the first normalized feature output by the previous level downsampling coding block; when the 3D frequency domain feedforward network module is integrated into the generator, the fourth input feature is the second activated feature output by the previous level upsampling generation block; The channel transformation layer, connected to the fourth 3D instance normalization layer, is used to transform the fourth normalized features in terms of channel dimensions and output features in the form of 3D tensors. An image patch serialization unit, connected to the channel transformation layer, is used to unfold the features in the form of a three-dimensional tensor into an image patch sequence; The second Fast Fourier Transform unit is connected to the image block serialization unit and is used to convert the image block sequence from the spatial domain to the frequency domain. A learnable frequency domain modulation unit is connected to the second fast Fourier transform unit, which is used to multiply the transformed features element-wise with a learnable weight matrix in the frequency domain and output the modulated spectrum. The second inverse fast Fourier transform unit is connected to the learnable frequency domain modulation unit and is used to convert the modulated spectrum from the frequency domain back to the spatial domain and output a spatial domain sequence. An image patch reconstruction unit, connected to the second inverse fast Fourier transform unit, is used to reconstruct the spatial domain sequence into reconstructed features in the form of a three-dimensional tensor. A gated activation layer, connected to the image patch reconstruction unit, is used to perform nonlinear activation on the reconstruction features in the form of three-dimensional tensors, outputting a first feedforward activation feature, and adding the first feedforward activation feature and the fourth input feature through a residual connection to form a second target frequency domain feedforward feature; wherein, when the three-dimensional frequency domain feedforward network module is integrated into the encoder, the second target frequency domain feedforward feature is the first frequency domain feedforward feature; when the three-dimensional frequency domain feedforward network module is integrated into the generator, the second target frequency domain feedforward feature is the second frequency domain feedforward feature.

4. The method for training a scalp-to-brain magnetic resonance imaging generative model according to claim 1, characterized in that, The composite loss function is a weighted sum of multiple losses, including: The first adversarial loss term is calculated based on the image discriminator's distinction between the synthetic brain MRI data and the real brain MRI sample, and is used to optimize the generator. The second adversarial loss term is calculated based on the distinction result between the latent feature representation and the random noise vector by the feature discriminator, and is used to optimize the encoder; The frequency domain loss term is obtained by converting the synthetic brain magnetic resonance imaging data and the real brain magnetic resonance imaging sample to the frequency domain and calculating the difference in frequency domain representation. The multi-scale spatial loss term is obtained by calculating the absolute error between the synthetic brain MRI data and the real brain MRI sample on image patches of different scales; The perceptual similarity loss term is obtained by comparing the difference between the depth feature representations of the synthetic brain magnetic resonance imaging data and the real brain magnetic resonance imaging samples. The feature matching loss term is obtained by calculating the difference between the features extracted by the intermediate multi-layer network layers of the encoder from the scalp magnetic resonance imaging sample and the synthetic brain magnetic resonance imaging data.

5. The method for training a scalp-to-brain magnetic resonance imaging generative model according to claim 4, characterized in that, The feature discriminator includes multiple fully connected layers for processing the latent feature representation and random noise vectors sampled from the preset probability distribution and outputting a first scalar score. The first scalar score is used to construct a second adversarial loss term in the composite loss function to constrain the distribution of the latent feature representation to be close to the preset probability distribution. Each fully connected layer of the feature discriminator is sequentially connected to a fifth 3D instance normalization layer and a fifth activation function layer.

6. The method for training a scalp-to-brain magnetic resonance imaging generative model according to claim 4, characterized in that, The image discriminator is a discriminator based on a three-dimensional convolutional neural network, used to process the synthetic brain magnetic resonance imaging data and the real brain magnetic resonance imaging samples and output a second scalar score. The second scalar score is used to construct the first adversarial loss term in the composite loss function, so that the first adversarial loss term is calculated based on the second scalar score, and a gradient penalty is introduced in the calculation process to follow the Wasserstein distance optimization criterion.

7. A method for generating scalp-to-brain magnetic resonance imaging, characterized in that, include: Acquire scalp magnetic resonance imaging data of the target object; The scalp magnetic resonance imaging data is input into the scalp-to-brain magnetic resonance imaging generation model, so that the scalp-to-brain magnetic resonance imaging generation model outputs corresponding synthetic brain magnetic resonance imaging data, wherein the scalp-to-brain magnetic resonance imaging generation model is trained based on the scalp-to-brain magnetic resonance imaging generation model training method as described in any one of claims 1 to 6.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the scalp-to-brain magnetic resonance imaging generation model training method as described in any one of claims 1 to 6, and / or implements the scalp-to-brain magnetic resonance imaging generation method as described in claim 7.

Images