Meg source reconstruction method and system based on pinn-vaes

Abstract

The application provides a MEG source reconstruction method and system based on a PINN-VAE, comprising the following steps: acquiring MEG original data, and processing the MEG original data to obtain MEG data of multiple time windows; inputting the MEG data of each time window into a pre-trained PINN-VAE hybrid neural network model to obtain target dipole parameters and a target predicted magnetic field corresponding to each time window; and mapping the target dipole parameters to a source space of a standard brain template. The application can inhibit physically unreasonable solutions and improve the interpretability of the MEG source reconstruction result.

Description

Technical Field

[0001] This invention relates to the field of neuroimaging data analysis technology, and in particular to a MEG source reconstruction method and system based on PINN-VAE. Background Technology

[0002] Magnetoencephalography (MEG) is a non-invasive neuroimaging technique that studies brain function by measuring the extremely weak magnetic field signals generated by neural activity in the brain. Source reconstruction aims to infer the location, direction, and intensity of intracranial neural current sources based on measurements from sensors outside the scalp; essentially, it is an inverse electromagnetic problem. Because different source distributions may produce similar external magnetic field observations, and the number of sensors is far less than the number of potential source points, and noise and modeling errors are amplified during inversion, this problem often exhibits non-unique solutions, underdetermined equations, and sensitivity to perturbations, leading to unstable results and difficulty in obtaining high-resolution, reliable estimates.

[0003] Traditional MEG source reconstruction methods are mainly divided into two categories: Equivalent Current Dipoles (ECD) fitting and Distributed Source Imaging (DSI) methods. The former is a parameterized method that assumes a small number of equivalent dipoles and fits the measurement signal through nonlinear optimization. This method is relatively straightforward but often requires pre-setting the number of dipoles and is sensitive to initial values, easily leading to biases when multiple sources are activated simultaneously. In distributed source imaging methods, the cerebral cortex is discretized into a large number of source points, and a large-scale underdetermined linear problem is solved. Common approaches include Minimum Norm Estimate (MNE), Low-Resolution Electromagnetic Tomography (LORETA), Beamforming, and Sparse Bayesian learning. These methods typically require careful selection of regularization intensity or prior hyperparameters, resulting in high computational costs and difficulty in balancing spatial resolution and robustness.

[0004] In recent years, data-driven deep learning methods have also emerged, such as using Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs) to directly learn the nonlinear mapping from magnetoencephalography (MEG) signals to source distributions, thereby achieving faster output during the inference stage. While these methods can improve speed and reconstruction quality in some scenarios, they often rely on large-scale labeled training data. However, obtaining real source locations in the field of MEG is difficult, resulting in insufficient training supervision information and limiting generalization ability. Furthermore, purely data-driven models lack explicit constraints on electromagnetic physics laws, potentially leading to physically unreasonable solutions and weak interpretability, making consistency verification and reliability assessment from a neuroscience perspective challenging. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a MEG source reconstruction method and system based on PINN-VAE, which can suppress physically unreasonable solutions and improve the interpretability of MEG source reconstruction results.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a MEG source reconstruction method based on PINN-VAE, comprising: acquiring raw MEG data and processing the raw MEG data to obtain MEG data for multiple time windows; inputting the MEG data of each time window into a pre-trained PINN-VAE hybrid neural network model to obtain the target dipole parameters and target predicted magnetic field corresponding to each time window; and mapping the target dipole parameters to the source space of a standard brain template.

[0007] Optionally, the PINN-VAE hybrid neural network model includes: an encoder, a resampling module, a decoder, and a dipole forward modeling module. MEG data from each time window are input into the pre-trained PINN-VAE hybrid neural network model to obtain the target dipole parameters and the target predicted magnetic field for each time window. This includes: inputting the MEG data from each time window into the encoder for feature extraction to obtain the target latent variable distribution parameters; inputting the target latent variable distribution parameters and the MEG data from each time window into the resampling module for reparameterization sampling to obtain sampled MEG data; inputting the sampled MEG data into the decoder for decoding to obtain the target dipole parameters for each time window; wherein, the target dipole parameters include: the target position of the target dipole and the target current dipole moment vector; inputting the target position of the target dipole and the target current dipole moment vector into the dipole forward modeling module for deduction to obtain the target predicted magnetic field.

[0008] Optionally, the sampled MEG data is input into a decoder for decoding to obtain the target dipole parameters corresponding to each time window, including: inputting the sampled MEG data into the decoder for decoding to obtain the initial position, initial unit direction vector, and initial intensity of the target dipole corresponding to each time window; constraining the initial position, initial unit direction vector, and initial intensity of the target dipole based on preset constraints to obtain the target position, target unit direction vector, and target intensity of the target dipole corresponding to each time window; and calculating the target current dipole moment vector of the target dipole based on the target unit direction vector and target intensity of the target dipole.

[0009] Optionally, the position of the target dipole and the target current dipole moment vector are input into the dipole forward modeling module for deduction to obtain the target predicted magnetic field, including: calculating the magnetic induction intensity of the target dipole at the MEG sensor position based on the target position and the target current dipole moment vector; superimposing the magnetic induction intensity of the target dipole at the sensor position in each time window to obtain the total magnetic field intensity; and calculating the projection of the total magnetic field intensity onto the MEG sensor direction vector to obtain the target predicted magnetic field.

[0010] Optionally, the target dipole parameters are mapped to the source space of the standard brain template, including: obtaining the source space file of the standard brain template, and obtaining the cortical vertex coordinate sets of the left hemisphere source space and the right hemisphere source space based on the source space file of the standard brain template to obtain the whole cortical vertex set; for each target dipole, calculating the spatial weight and orientation consistency weight of the whole cortical vertex based on the target position of the target dipole and the whole cortical vertex set; obtaining the contribution of the target dipole to the whole cortical vertex based on the spatial weight, orientation consistency weight and target current dipole moment vector of the whole cortical vertex; summing the contributions of each target dipole within the same time window and multiplying by a global scaling factor to obtain the whole vertex activity distribution vector corresponding to the time window; splitting the whole vertex activity distribution vector into the left hemisphere activity matrix and the right hemisphere activity matrix according to the preset vertex index to obtain the temporal activity data and generate the source time process file.

[0011] Optionally, before obtaining the raw MEG data, the process further includes: obtaining the raw MEG training data and processing it to obtain MEG training data for multiple time windows; inputting the MEG training data for each time window into the PINN-VAE hybrid neural network model to obtain the sample latent variable distribution parameters of the MEG training data, as well as the sample dipole parameters and sample predicted magnetic fields corresponding to each time window; calculating the multi-objective loss function value based on the sample latent variable distribution parameters of the MEG training data, the sample dipole parameters, sample predicted magnetic fields, and MEG training data corresponding to each time window; and updating the model parameters of the PINN-VAE hybrid neural network model based on the multi-objective loss function value.

[0012] Optionally, based on the sample latent variable distribution parameters of the MEG training data, the sample dipole parameters corresponding to each time window, the sample predicted magnetic field, and the MEG training data, the multi-objective loss function value is calculated, including: calculating the VAE reconstruction loss value based on the MEG training data and MEG reconstruction data; calculating the KL divergence loss value based on the sample latent variable distribution parameters of the MEG training data; calculating the forward consistency loss value based on the sample predicted magnetic field and the true magnetic field; and calculating the position and depth regularization loss value based on the sample dipole parameters. The forward consistency loss value and the position and depth regularization loss value are then weighted to obtain the physical residual loss value. Finally, the VAE reconstruction loss value, the KL divergence loss value, and the physical residual loss value are weighted to obtain the multi-objective loss function value.

[0013] Secondly, the present invention provides a MEG source reconstruction system based on PINN-VAE, comprising: a data processing module for acquiring raw MEG data and processing the raw MEG data to obtain MEG data for multiple time windows; a reconstruction module for inputting the MEG data of each time window into a pre-trained PINN-VAE hybrid neural network model to obtain the target dipole parameters and the target predicted magnetic field corresponding to each time window; and a mapping module for mapping the target dipole parameters to the source space of a standard brain template.

[0014] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the steps of the method provided in any of the first aspects above.

[0015] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, performs the steps of the method provided in any of the first aspects above.

[0016] This invention brings the following beneficial effects: The MEG source reconstruction method and system based on PINN-VAE provided by this invention first acquires raw MEG data and processes it to obtain MEG data for multiple time windows. Then, the MEG data for each time window is input into a pre-trained PINN-VAE hybrid neural network model to obtain the target dipole parameters and the target predicted magnetic field corresponding to each time window. Finally, the target dipole parameters are mapped to the source space of a standard brain template. In this method, processing the raw MEG data improves the signal-to-noise ratio and quality of the input data; decoding the MEG data using a pre-trained PINN-VAE hybrid neural network model and outputting the target dipole parameters and the target predicted magnetic field, and mapping the target dipole parameters to the source space of a standard brain template, improves the efficiency of MEG source reconstruction, suppresses physically unreasonable solutions, and enhances the interpretability of the MEG source reconstruction results.

[0017] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 A flowchart of a MEG source reconstruction method based on PINN-VAE provided for embodiments of the present invention; Figure 2 A schematic diagram of the channel correlation coefficient between real sensor readings and model physical decoding readings provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the changes in total loss, reconstruction loss, KL loss, and physical loss with the number of training iterations, provided as an embodiment of the present invention. Figure 4 This is a schematic diagram showing the position and orientation of a sensor array and a reconstructed dipole drawn in a head coordinate system, as provided in an embodiment of the present invention. Figure 5A visual evoked response event curve diagram of raw data and reconstructed data provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of timing rendering in the fsaverage template using STC, provided by an embodiment of the present invention. Figure 7 A schematic diagram of a MEG source reconstruction system based on PINN-VAE provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Current data-driven deep learning methods, such as using convolutional neural networks (CNNs) or recurrent neural networks (RNNs), directly learn the nonlinear mapping from magnetoencephalography (MEG) signals to source distributions, thereby achieving faster output during the inference phase. While these methods can improve speed and reconstruction quality in some scenarios, they often rely on large-scale labeled training data. However, obtaining real source locations in the field of MEG is difficult, resulting in insufficient training supervision information and limiting generalization ability. Furthermore, purely data-driven models lack explicit constraints on electromagnetic physics, potentially leading to physically unreasonable solutions and weak interpretability, making consistency verification and reliability assessment from a neuroscience perspective challenging.

[0023] Based on this, the MEG source reconstruction method and system based on PINN-VAE provided in this embodiment of the invention can suppress physically unreasonable solutions and improve the interpretability of MEG source reconstruction results.

[0024] To facilitate understanding of this embodiment, a detailed description of a MEG source reconstruction method based on PINN-VAE disclosed in this invention will be provided first. This method can be executed by electronic devices, such as smartphones, computers, and tablets. See also... Figure 1 The flowchart shown illustrates a MEG source reconstruction method based on PINN-VAE, which mainly includes the following steps S101 to S103: Step S101: Obtain raw MEG data and process it to obtain MEG data for multiple time windows.

[0025] In one implementation, the raw MEG data is first loaded, bandpass filtered, and artifact components in the filtered MEG data are removed by independent component analysis to obtain initial MEG data. Then, the initial MEG data is segmented by a sliding window of a preset size, and the data in each window is standardized to obtain MEG data for multiple time windows.

[0026] In practice, the original MEG data file (.fif format) is loaded. When the number of channels in the original data is 97, the first 96 channels are selected. The original signal is bandpass filtered (1–80 Hz), and the ICA (Independent Component Analysis) denoising module is used to perform independent component decomposition on the original MEG data to remove high variance artifacts and obtain a high-quality MEG signal, i.e., the initial MEG data.

[0027] Furthermore, a sliding window method is used to divide the continuous MEG time series (initial MEG data) into fixed-length time segments. The time window size (e.g., 50 time points, corresponding to 50ms at a sampling rate of 1000 Hz) and step size (e.g., 25 time points) are set to generate MEG data for multiple time windows. The MEG data of each window are then Z-score standardized to make the mean of each channel 0 and the standard deviation 1.

[0028] Step S102: Input the MEG data of each time window into the pre-trained PINN-VAE hybrid neural network model to obtain the target dipole parameters and target predicted magnetic field corresponding to each time window.

[0029] In one implementation, the PINN-VAE hybrid neural network model employs a network architecture based on a deep fusion of a Physics-Informed Neural Network (PINN) and a Variational Autoencoder (VAE). The PINN-VAE model uses an encoder to extract the temporal features of MEG data for each time window and outputs latent variable distribution parameters. Reparameterized sampling is used to obtain the latent representation of the MEG data. Finally, decoding is performed to obtain the target dipole parameters corresponding to each time window, and a theoretical magnetic field (i.e., the target predicted magnetic field) is generated via a dipole forward modeling module. Simultaneously, the correlation coefficient between the target predicted magnetic field obtained from the forward modeling of the output target dipole parameters and the measured magnetic field is calculated as a result reliability index.

[0030] Step S103: Map the target dipole parameters to the source space of the standard brain template.

[0031] In one implementation, the reconstructed target dipole parameters are mapped to the source space of the standard brain template (fsaverage), and spatial smoothing is performed using a Gaussian kernel function to generate a Source Time Course (STC) file for subsequent time-frequency analysis.

[0032] The MEG source reconstruction method based on PINN-VAE provided in this invention improves the signal-to-noise ratio and quality of the input data by processing the raw MEG data; it decodes the MEG data using a pre-trained PINN-VAE hybrid neural network model and outputs the target dipole parameters and the target predicted magnetic field, and maps the target dipole parameters to the source space of the standard brain template, which can improve the efficiency of MEG source reconstruction, suppress physically unreasonable solutions, and improve the interpretability of MEG source reconstruction results.

[0033] In one implementation, the PINN-VAE hybrid neural network model includes: an encoder, a resampling module, a decoder, and a dipole forward modeling module. Regarding the aforementioned step S102, i.e., when inputting the MEG data of each time window into the pre-trained PINN-VAE hybrid neural network model to obtain the target dipole parameters and the target predicted magnetic field corresponding to each time window, the following methods can be used, including but not limited to: First, MEG data from each time window are input into the encoder for feature extraction to obtain the target latent variable distribution parameters; among which, the target latent variable distribution parameters include: the target mean vector and the target log-variance vector.

[0034] In practice, the encoder employs a variational autoencoder to extract features from the MEG data within the input time window and outputs the target latent variable distribution parameters: the target mean vector. With the target log-variance vector The dimension of the target latent variable distribution parameters is Where D is the number of dipoles. denoted as the standard deviation of the latent variable distribution.

[0035] Then, the target latent variable distribution parameters and MEG data for each time window are input into the resampling module for reparameterized sampling to obtain sampled MEG data.

[0036] In practical implementation, based on the target mean vector With the target log-variance vector Reparameterized sampling is performed according to the following formula:

[0037] in, To sample MEG data, This is a random noise vector sampled from a standard normal distribution. It follows a multivariate standard normal distribution, where the mean vector is the zero vector and the covariance matrix is ​​the identity matrix.

[0038] Next, the sampled MEG data is input into the decoder for decoding to obtain the target dipole parameters corresponding to each time window; among which, the target dipole parameters include: the target position of the target dipole, the target unit direction vector, the target intensity, and the target current dipole moment vector.

[0039] In practical implementation, the input to the decoder is The output is D target dipole parameters. For the th j A target dipole, the output target dipole parameters include: the target position of the target dipole. Target unit direction vector Target strength and the target current dipole moment vector .

[0040] In one implementation, to suppress physically unreasonable solutions and improve the interpretability of MEG source reconstruction results, constraints can be imposed on the decoder output to ensure that the output target dipole parameters are within a reasonable range. Specifically, firstly, the sampled MEG data is input into the decoder for decoding to obtain the initial position, initial unit direction vector, and initial intensity of the target dipole for each time window; then, based on preset constraints, the initial position, initial direction vector, and initial intensity of the target dipole are constrained to obtain the target position, target unit direction vector, and target intensity of the target dipole for each time window; finally, based on the target unit direction vector and target intensity of the target dipole, the target current dipole moment vector of the target dipole is calculated.

[0041] In practical implementation, the constraints include: (1) Position constraints (restricting the position of the dipole to within the head radius):

[0042] in, For the first j The target position of a dipole. For the first j The position before the dipole is constrained, i.e., the initial position; The effective radius of the head, The radius is the head radius.

[0043] (2) Direction constraint (obtaining the unit direction vector):

[0044] in, For the first j The target unit direction vector of each dipole For the first j The direction vector before the dipole constraint, i.e., the initial direction vector. It is a very small positive constant, usually taken as... or .

[0045] (3) Strength constraints (mapped to a physiologically reasonable range):

[0046] in, For the first j The target strength of a dipole This represents the lower limit of the physiologically reasonable range. This represents the upper limit of the physiologically reasonable range. For the first j The strength before the dipole constraint, i.e., the initial strength. To achieve the desired effect under the sigmoid activation function, Mapped to .

[0047] (4) Current dipole moment vector (used for forward modeling):

[0048] in, For the first j The vector of the current dipole moments of a dipole, in units .

[0049] Finally, the target position and target current dipole moment vector of the target dipole are input into the dipole forward modeling module for derivation to obtain the target predicted magnetic field.

[0050] In practical implementation, the dipole forward modeling module inputs the time windows based on the dipole Biot–Savart forward model. In addition to the spatial position and orientation information of the MEG sensor, the predicted magnetic field of the target is calculated. .

[0051] In one implementation, under the quasi-static approximation, each neuron is equivalent to a dipole, and the forward modeling of the dipole according to the Biot-Savart dipole model includes the following process: (1) Calculate the magnetic induction intensity of the target dipole at the MEG sensor location based on the target position and the target current dipole moment vector of the target dipole.

[0052] In practical implementation, for those located at the target location And the target current dipole moment vector is The dipole, whose magnetic flux density at MEG sensor position r. for:

[0053] in, , where is the vacuum permeability.

[0054] (2) The magnetic induction intensity of the target dipole at the sensor position within each time window is superimposed to obtain the total magnetic field intensity.

[0055] In practice, the total magnetic field strength within each time window It satisfies linear superposition, that is:

[0056] (3) Calculate the projection of the total magnetic field intensity onto the direction vector of the MEG sensor to obtain the target predicted magnetic field.

[0057] In practical implementation, the MEG sensor reading is the magnetic field vector in the sensor direction. The projection on, that is:

[0058] In one implementation, for the aforementioned step S103, i.e., when mapping the target dipole parameters to the source space of the standard brain template, the following methods can be used, including but not limited to the following steps: Steps 1 to 5: Step 1: Obtain the source space file of the standard brain template, and based on the source space file of the standard brain template, obtain the cortical vertex coordinate sets of the left hemisphere source space and the right hemisphere source space respectively, to obtain the whole cortical vertex set.

[0059] In practical implementation, MNE-Python is used to read the source space file fsaverage-ico-5-src.fif of the standard brain template fsaverage, and to obtain the sets of cortical vertex coordinates for the left and right hemisphere source spaces, respectively, denoted as... and And merge them into a set of vertices of the entire cortex. .

[0060] Furthermore, the target position of the target dipole is aligned with the vertex coordinates of the entire cortical vertex set. In specific implementation, for the data of each sliding time window, the target dipole parameters output by the PINN-VAE hybrid neural network model are obtained, including the target position of each target dipole and the target current dipole moment vector. The target position of the target dipole and the vertex coordinates of the entire cortical vertex set are both represented using the same head coordinate system, thus allowing for direct Euclidean distance calculation.

[0061] Step 2: For each target dipole, calculate the spatial weight and orientation consistency weight of the whole cortex vertices based on the target position of the target dipole and the whole cortex vertex set.

[0062] Specifically, calculating the spatial weights and orientation consistency weights of all cortical vertices includes the following steps 21 to 22: Step 21: For each target dipole, based on the target position of the target dipole and the set of vertices of the whole cortex, calculate the Euclidean distance between the target dipole and the vertices of the whole cortex. clan Distance is calculated, and spatial weights of all cortical vertices are generated based on Gaussian kernel and Euclidean distance.

[0063] In practical implementation, for any target dipole, its position is calculated. To the apex of the entire cortex The Euclidean distance is calculated, and the spatial weights of all cortical vertices are generated using a Gaussian kernel. :

[0064] in, is the Gaussian kernel scale parameter, used to control the spatial influence range of the dipole on the surrounding vertices.

[0065] Step 22: For each target dipole, calculate the vertex direction vector based on the target position of the target dipole and the set of vertices in the whole cortex, and obtain the orientation consistency weight of the vertices in the whole cortex based on the vertex direction vector and the current dipole moment vector of the target dipole.

[0066] In practical implementation, to reflect the influence of the current dipole moment direction of the target dipole on the contribution of the entire cortex vertex, the relative direction vector of the vertex is calculated. :

[0067] And the vertex relative direction vector and the current dipole moment unit vector Perform a dot product and take the non-negative part to form the directional consistency weights: .

[0068] Step 3: Based on the spatial weights, directional consistency weights, and target current dipole moment vectors of the full cortex vertices, obtain the contribution of the target dipole to the full cortex vertices.

[0069] In practical implementation, for any target dipole, its position at the full cortical vertex v The contribution is :

[0070] Step 4: Sum the contributions of each target dipole within the same time window and multiply by the global scaling factor to obtain the full vertex activity distribution vector corresponding to the time window.

[0071] In practice, the contributions of multiple target dipoles within the same time window are summed to obtain the full vertex activity distribution vector corresponding to that time window. Furthermore, to improve numerical display and visualization, the full vertex activity distribution vector is multiplied by a global scaling factor.

[0072] Step 5: Split the full vertex activity distribution vector into a left hemisphere activity matrix and a right hemisphere activity matrix according to the preset vertex index to obtain the time-series activity data and generate the source time process file.

[0073] In practice, the full vertex activity distribution vector is split into a left hemisphere activity matrix and a right hemisphere activity matrix according to the vertex index, forming a shape as follows: Temporal activity data. A set of vertex indexes based on the fsaverage source space. and We construct mne.SourceEstimate objects for the left and right hemispheres respectively and save them as source time process (STC) files for visualization and subsequent analysis.

[0074] Set the start time when building SourceEstimate. With time step .in, Retrieve the epoch start time parameter of the data. Take the reciprocal of the sampling frequency This generates an STC time axis description that matches the window sequence length.

[0075] The PINN-VAE hybrid neural network model used in this embodiment of the invention was trained in the following manner: First, the raw MEG training data is obtained and processed to obtain MEG training data for multiple time windows.

[0076] Then, the MEG training data for each time window is input into the PINN-VAE hybrid neural network model to obtain the sample latent variable distribution parameters of the MEG training data, as well as the sample dipole parameters and sample predicted magnetic fields corresponding to each time window.

[0077] The sample dipole parameters include: the sample position, the sample unit direction vector, the sample intensity, and the sample current dipole moment vector.

[0078] In specific implementation, the process of processing the original MEG training data and outputting the sample latent variable distribution parameters, sample dipole parameters and sample predicted magnetic fields corresponding to each time window through the PINN-VAE hybrid neural network model are similar to the aforementioned embodiments, and will not be repeated here.

[0079] Next, based on the sample latent variable distribution parameters of the MEG training data, as well as the sample dipole parameters, sample predicted magnetic field, and MEG training data corresponding to each time window, the multi-objective loss function value is calculated.

[0080] In practical implementation, a multi-objective loss function is defined, and the VAE reconstruction loss, KL divergence loss, and physical residual loss are jointly constructed. The multi-objective loss function values ​​are calculated, including: (1) Calculate the VAE reconstruction loss value based on MEG training data and MEG reconstruction data.

[0081] VAE reconstruction loss is used to measure the difference between the reconstructed signal (i.e., the sample predicted magnetic field) and the high-quality initial MEG signal after ICA processing (i.e., MEG training data). In this embodiment of the invention, mean squared error (MSE) is used to calculate the VAE reconstruction loss value. :

[0082] in, N This represents the number of samples in a window within a batch. C For the number of channels of the MEG sensor, W The time window length, This is the original signal (i.e., MEG training data). For signal reconstruction.

[0083] (2) Calculate the KL divergence loss value based on the latent variable distribution parameters of the MEG training data.

[0084] KL divergence loss is used to constrain the posterior distribution of parameters of the latent distribution variables of a sample. q ( z | x Approximate prior distribution p ( z )=N (0, I KL divergence loss value The parsing form is:

[0085] in, L As a potential spatial dimension, and These are the mean and logarithmic variance vectors of the encoder output of the PINN-VAE hybrid neural network model, respectively.

[0086] (3) Based on the sample predicted magnetic field and the real magnetic field, calculate the forward consistency loss value, and based on the sample dipole parameters, calculate the position and depth regularization loss value, and weight the forward consistency loss value and the position and depth regularization loss value to obtain the physical residual loss value.

[0087] The physical residual loss consists of two parts: forward consistency loss and location-depth canonicalization loss. Specifically, the theoretical magnetic field generated by the dipole (i.e., the sample predicted magnetic field) is calculated according to the Biot-Savart law, and then compared with the actually measured MEG signal (measured by the sensor) to obtain the forward consistency loss value. :

[0088] in, The magnetic field distribution, calculated using a physical forward model based on the sample dipole parameters obtained from the decoder, is the sample predicted magnetic field. This is the high-quality initial MEG signal after ICA denoising, i.e., the real magnetic field.

[0089] Furthermore, a position amplitude regularization term is set to penalize sample dipoles that are too far from the origin, resulting in a position depth regularization loss value. :

[0090] in, It is the number of sample dipoles within the time window. It is the first i The first time window sample j The three-dimensional position of the sample dipole.

[0091] Physical residual loss value The general form is:

[0092] (4) The VAE reconstruction loss value, KL divergence loss value and physical residual loss value are weighted and calculated to obtain the multi-objective loss function value.

[0093] Specifically, the multi-objective loss function value Using a weighted sum:

[0094] Wherein, λ1=1.0 (VAE reconstruction loss weight), λ2=0.01 (KL divergence weight), and λ3=0.1 (physical constraint weight).

[0095] Finally, the model parameters of the PINN-VAE hybrid neural network model are updated based on the multi-objective loss function value.

[0096] In practice, a warmup training strategy is implemented and model parameters are optimized. The total number of warmup epochs is set to warmup_epochs (e.g., 200). After warmup_epochs, physical loss is introduced with a fixed weight of 0.1.

[0097] The MEG source reconstruction method based on PINN-VAE provided in this invention combines the powerful data fitting capabilities of deep learning with prior knowledge of electromagnetic field physics laws. During the training process of the neural network, it simultaneously optimizes data reconstruction errors and physical residuals, achieving an organic unity between data-driven and physical constraints. Compared with traditional pure optimization inversion methods, this invention is more efficient in the inference stage and can output dipole parameters and their uncertainty metrics. Compared with purely data-driven deep learning methods, this invention explicitly introduces electromagnetic physics constraints, which can suppress physical inconsistencies and improve the model's generalization ability.

[0098] For ease of understanding, this embodiment of the invention uses raw MEG data from a visual stimulus task as input and employs a PINN-VAE-based magnetoencephalography (MEG) source reconstruction method to achieve a complete process from sensor observation to dipole parameter estimation, and further mapping to the fsaverage template source space to generate an STC file, including the following steps S1 to S8: Step S1: Data loading and preprocessing.

[0099] Specifically, the raw MEG data (.fif format) is read to obtain the sensor time-series signal matrix. The raw MEG signal is bandpass filtered between 1 and 80 Hz to suppress DC drift and high-frequency noise. Independent component decomposition (ICA) is performed on the filtered signal, and the top 30% of high-variance components are removed based on the variance of the component time series to obtain a clean MEG signal. Simultaneously, the sensor's spatial position (x, y, z) is read from sensor_coordinates.csv.

[0100] Step S2: Sliding window segmentation and data standardization.

[0101] Specifically, events are constructed at fixed intervals on the timeline and epochs are captured (e.g., by setting...). , Then, multiple epochs are concatenated along the time dimension to form a continuous sequence for sliding window sample generation. The window size is set to window_size=50 (50 sampling points) and stride=25. The continuous time series is then segmented using a sliding window to obtain the training sample set. Let one window sample be: ,in, C The number of channels is (96). W The window length is 50. The samples are Z-score standardized to obtain the standardized input. This is to improve the stability of network training.

[0102] Step S3: PINN-VAE model construction.

[0103] S31: Encoder (Temporal Convolutional Feature Extraction + Latent Variable Distribution): The input is window data (N, C, 50), which is processed by convolution and pooling to obtain global features. Then, the latent variable distribution parameters are output through two fully connected branches. and .

[0104] S32, Reparameterized Sampling: .

[0105] S33. Decoder outputs multiple dipole parameters and applies constraints: The decoder outputs the parameters of the D dipoles, and applies constraints to the parameters of the first dipole. j Each dipole, the output includes: position Unit direction vector With strength And apply the following constraints: (a) Positional constraints (limited to the head radius):

[0106] (b) Direction constraint (obtaining the unit direction vector):

[0107] (c) Strength constraints (mapped to a physiologically reasonable range):

[0108] (d) Current dipole moment vector (for forward modeling):

[0109] S34: Dipole orthogonal model.

[0110] D dipoles in each window By inputting the dipole Biot-Savart forward modeling module and combining the sensor coordinates and direction vectors, the predicted magnetic field is obtained. This is used for calculating the forward consistency term in the subsequent loss function. The Biot-Savart dipole forward model involves treating each neuron as an equivalent dipole under a quasi-static approximation. For neurons located at... And the current dipole moment is The dipole, at the sensor position The magnetic field strength at that location is:

[0111] The total magnetic field satisfies linear superposition:

[0112] MEG sensor readings are the magnetic field vectors in the sensor's direction. Projection on:

[0113] Step S4: Loss function construction.

[0114] Specifically, the multi-objective loss function is: .

[0115] (a) VAE reconstruction loss (Error in reconstructing window signal):

[0116] in, N The number of window samples within a batch. C Number of sensor channels W This is the window length. The standard MEG data obtained in step S2, The reconstructed MEG data obtained in step S3.

[0117] (b) KL divergence (VAE regular expression):

[0118] Where L is the latent spatial dimension (latent_dim). and These are the mean and logarithmic variance vectors of the encoder output, respectively.

[0119] (c) Physical residual loss Includes: forward consistency loss Location and depth regularization loss .

[0120] 1) Forward consistency loss :

[0121] 2) Position depth regularization (Penalize the dipole being too far from the center of the head):

[0122] Therefore, . Among them, is the physical loss scale coefficient (for numerical stability), is the regularization weight.

[0123] Step S5: Training strategy (warmup and fixed physical weight).

[0124] 1) Set warmup_epochs = 200; 2) When epoch < warmup_epochs, do not introduce physical loss; 3) When epoch >= warmup_epochs, introduce physical loss, and the physical weight is fixed at 0.1.

[0125] Step S6: Dipole parameter inference and rationality constraint.

[0126] Input the MEG data of the test window into the model, and output multiple dipole parameters corresponding to each window: position and dipole moment vector . Among them: 1) The position is constrained within the head range by tanh and brain_radius 0.5; 2) The magnitude of the dipole moment is mapped to a physiologically reasonable range by sigmoid ; 3) At the same time, calculate the error between the predicted magnetic field and the measured magnetic field (such as MSE or correlation coefficient) to evaluate the reconstruction quality of this window.

[0127] Step S7: Map to the fsaverage source space and generate STC.

[0128] S71: Load the fsaverage source space: Read fsaverage-ico-5-src.fif, and merge the vertex coordinates of the left and right hemispheres into a vertex set V .

[0129] S72: Calculate the spatial weight and direction weight: For each window and each dipole ( ), for any vertex calculate: a) Spatial weights (Gaussian kernel): .

[0130] b) Directional consistency weight: .

[0131] c) Vertex contribution: .

[0132] The source space activity distribution of a window is obtained by summing multiple dipoles within the same window.

[0133] S73: Construct and save SourceEstimate: Split the obtained (n_vertices, n_windows) activity matrix into left and right hemispheres, and construct mne.SourceEstimate, where the time step is: Save it as a .stc file for later visualization and analysis.

[0134] Step S8: Effect evaluation and visualization.

[0135] For details, see Figure 2 The diagram shown illustrates the channel correlation coefficient between real sensor readings and model physical decoding readings. (See attached diagram.) Figure 3 The diagram shown illustrates the changes in total loss, reconstruction loss, KL loss, and physical loss over the number of training iterations. (See also...) Figure 4 The diagram shown illustrates the position and orientation of the sensor array and the reconstructed dipole plotted in a head coordinate system. (See attached image.) Figure 5 The image shows visual evoked response event curves for raw and reconstructed data. The top graph shows the result for the raw data, and the bottom graph shows the result for the reconstructed data. (See also...) Figure 6 The diagram shows a time-series rendering of STC in the fsaverage template.

[0136] The method provided in this embodiment of the invention employs a VAE framework for probabilistic modeling, encoding high-dimensional MEG time series data into a low-dimensional latent space representation, decoding dipole parameters (position, orientation, and intensity), and achieving end-to-end backpropagation training through reparameterization techniques. A physical residual term based on Maxwell's equations and Biot-Savart's law is explicitly introduced into the loss function to constrain the magnetic field generated by the reconstructed dipole configuration to satisfy electromagnetic field physics laws, including zero magnetic field divergence and the coupling relationship between the electric and magnetic fields. An independent component analysis (ICA) preprocessing module automatically... This invention identifies and removes physiological artifacts such as eye movement, electrocardiogram, and electromyography in MEG signals, improving the signal-to-noise ratio and quality of the input data. A Warmup training strategy is employed, reducing the weight of physical constraints in the early stages of training and gradually increasing it to the target value to avoid training instability or local convergence caused by premature dominance of physical terms. A complete process from raw MEG signals to a standard brain template source space is implemented, including fully automated processing of data preprocessing, model training, dipole extraction, source space mapping, and visualization analysis. Uncertainty quantification is provided through a probabilistic framework, outputting confidence intervals for each reconstructed source, providing reliability assessment for clinical decision-making. Compared to traditional pure optimization inversion methods, this invention is more efficient in the inference stage and can output dipole parameters and their uncertainty measures. Compared to purely data-driven deep learning methods, this invention explicitly introduces electromagnetic physical constraints, which can suppress physical inconsistencies and improve the model's generalization ability.

[0137] In addition to the PINN-VAE-based MEG source reconstruction method provided in the foregoing embodiments, this invention also provides a PINN-VAE-based MEG source reconstruction system, see [link to relevant documentation]. Figure 7 The diagram shown illustrates the structure of a MEG source reconstruction system based on PINN-VAE, showing that the system mainly includes the following components: The data processing module 701 is used to acquire raw MEG data and process the raw MEG data to obtain MEG data for multiple time windows. The reconstruction module 702 is used to input the MEG data of each time window into the pre-trained PINN-VAE hybrid neural network model to obtain the target dipole parameters and the target predicted magnetic field corresponding to each time window. Mapping module 703 is used to map the target dipole parameters to the source space of the standard brain template.

[0138] The MEG source reconstruction system based on PINN-VAE provided in this embodiment of the invention improves the signal-to-noise ratio and quality of the input data by processing the raw MEG data; it decodes the MEG data using a pre-trained PINN-VAE hybrid neural network model and outputs the target dipole parameters and the target predicted magnetic field, and maps the target dipole parameters to the source space of the standard brain template, which can improve the efficiency of MEG source reconstruction, suppress physically unreasonable solutions, and improve the interpretability of MEG source reconstruction results.

[0139] It should be noted that the system provided in this embodiment of the invention has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the system embodiment can be referred to the corresponding content in the aforementioned method embodiment. The specific numerical values ​​provided in the implementation of this invention are merely exemplary and are not intended to limit the scope of the invention.

[0140] This invention also provides an electronic device, specifically, the electronic device includes a processor and a storage device; the storage device stores a computer program, and the computer program, when run by the processor, executes the method described in any of the above embodiments.

[0141] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 100 includes: a processor 80, a memory 81, a bus 82, and a communication interface 83. The processor 80, the communication interface 83, and the memory 81 are connected through the bus 82. The processor 80 is used to execute executable modules, such as computer programs, stored in the memory 81.

[0142] The memory 81 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 83 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0143] Bus 82 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0144] The memory 81 is used to store programs. After receiving an execution instruction, the processor 80 executes the program. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 80 or implemented by the processor 80.

[0145] The processor 80 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 80 or by instructions in software form. The processor 80 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 81. The processor 80 reads the information in memory 81 and, in conjunction with its hardware, completes the steps of the above method.

[0146] The computer program product of the readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0147] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory, magnetic disks, or optical disks.

[0148] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A MEG source reconstruction method based on PINN-VAE, characterized in that, include: Obtain raw MEG data and process the raw MEG data to obtain MEG data for multiple time windows; The MEG data of each time window is input into a pre-trained PINN-VAE hybrid neural network model to obtain the target dipole parameters and target predicted magnetic field corresponding to each time window. The target dipole parameters are mapped to the source space of the standard brain template.

2. The method according to claim 1, characterized in that, The PINN-VAE hybrid neural network model includes: an encoder, a resampling module, a decoder, and a dipole forward modeling module; The MEG data for each time window are input into a pre-trained PINN-VAE hybrid neural network model to obtain the target dipole parameters and the target predicted magnetic field corresponding to each time window, including: The MEG data of each time window is input into the encoder for feature extraction to obtain the target latent variable distribution parameters; The target latent variable distribution parameters and MEG data for each time window are input into the resampling module for reparameterized sampling to obtain sampled MEG data. The sampled MEG data is input into the decoder for decoding to obtain the target dipole parameters corresponding to each time window; wherein, the target dipole parameters include: the target position of the target dipole and the target current dipole moment vector; The target position and target current dipole moment vector of the target dipole are input into the dipole forward modeling module for deduction to obtain the target predicted magnetic field.

3. The method according to claim 2, characterized in that, The sampled MEG data is input into the decoder for decoding to obtain the target dipole parameters corresponding to each time window, including: The sampled MEG data is input into the decoder for decoding to obtain the initial position, initial unit direction vector and initial intensity of the target dipole corresponding to each time window; The initial position, initial unit direction vector, and initial intensity of the target dipole are constrained based on preset constraints to obtain the target position, target unit direction vector, and target intensity of the target dipole corresponding to each time window. Based on the target unit direction vector and target intensity of the target dipole, the target current dipole moment vector of the target dipole is calculated.

4. The method according to claim 2, characterized in that, The position of the target dipole and the target current dipole moment vector are input into the dipole forward modeling module for deduction to obtain the target predicted magnetic field, including: Based on the target position of the target dipole and the target current dipole moment vector, calculate the magnetic induction intensity of the target dipole at the MEG sensor position; The total magnetic field strength is obtained by superimposing the magnetic induction intensity of the target dipole at the sensor position within each time window. The projection of the total magnetic field intensity onto the direction vector of the MEG sensor is calculated to obtain the target predicted magnetic field.

5. The method according to claim 2, characterized in that, Mapping the target dipole parameters to the source space of a standard brain template includes: Obtain the source space file of the standard brain template, and based on the source space file of the standard brain template, obtain the cortical vertex coordinate sets of the left hemisphere source space and the right hemisphere source space respectively, to obtain the whole cortical vertex set; For each target dipole, the spatial weight and orientation consistency weight of the full cortical vertex are calculated based on the target position of the target dipole and the full cortical vertex set. The contribution of the target dipole to the full cortical vertex is obtained based on the spatial weight, orientation consistency weight, and target current dipole moment vector of the full cortical vertex. The contributions of each target dipole within the same time window are summed and multiplied by a global scaling factor to obtain the full vertex activity distribution vector corresponding to the time window. The full vertex activity distribution vector is split into a left hemisphere activity matrix and a right hemisphere activity matrix according to a preset vertex index to obtain time-series activity data and generate a source time process file.

6. The method according to any one of claims 1-5, characterized in that, Before obtaining the raw MEG data, the following steps are also included: Obtain the raw MEG training data and process the raw MEG training data to obtain MEG training data for multiple time windows; The MEG training data of each time window is input into the PINN-VAE hybrid neural network model to obtain the sample latent variable distribution parameters of the MEG training data, as well as the sample dipole parameters and sample predicted magnetic fields corresponding to each time window. Based on the sample latent variable distribution parameters of the MEG training data, the sample dipole parameters corresponding to each time window, the sample predicted magnetic field, and the MEG training data, the multi-objective loss function value is calculated. Based on the multi-objective loss function value, the model parameters of the PINN-VAE hybrid neural network model are updated.

7. The method according to claim 6, characterized in that, Based on the sample latent variable distribution parameters of the MEG training data, the sample dipole parameters corresponding to each time window, the sample predicted magnetic field, and the MEG training data, the multi-objective loss function value is calculated, including: Based on the MEG training data and MEG reconstruction data, calculate the VAE reconstruction loss value; Based on the sample latent variable distribution parameters of the MEG training data, calculate the KL divergence loss value; Based on the predicted magnetic field and the actual magnetic field of the sample, the forward consistency loss value is calculated, and based on the dipole parameters of the sample, the position and depth regularization loss value is calculated. The forward consistency loss value and the position and depth regularization loss value are weighted and calculated to obtain the physical residual loss value. The VAE reconstruction loss value, the KL divergence loss value, and the physical residual loss value are weighted and calculated to obtain the multi-objective loss function value.

8. A MEG source reconstruction system based on PINN-VAE, characterized in that, include: The data processing module is used to acquire raw MEG data and process the raw MEG data to obtain MEG data for multiple time windows. The reconstruction module is used to input the MEG data of each time window into the pre-trained PINN-VAE hybrid neural network model to obtain the target dipole parameters and target predicted magnetic field corresponding to each time window; The mapping module is used to map the target dipole parameters to the source space of the standard brain template.

9. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program, when run by a processor, performs the steps of the method described in any one of claims 1 to 7.

Images