Model training method and device for magnetoencephalography system and source localization method

Abstract

The present disclosure relates to a model training method, device and a source localization method for a magnetoencephalography system. The model training method comprises: obtaining a three-dimensional model of a brain of a subject and a relative position relationship between the brain and a sensor array; performing grid division on a preset region of interest in the three-dimensional model to obtain a plurality of simulation source points; generating at least one set of simulation source signals at the plurality of simulation source points in the three-dimensional model; determining, based on the at least one set of simulation source signals, at least one set of simulation sensor signals expected to be collected by the sensor array using a forward model, wherein the forward model is a mathematical model representing a relationship between electromagnetic activity of a source point in the brain and magnetoencephalography signals collected by the sensor array, and the forward model is determined based on the relative position relationship; and training a deep learning model to construct a magnetoencephalography source localization model by taking the at least one set of simulation sensor signals as input and the at least one set of simulation source signals as labels.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of medical devices, and in particular to a model training method and a model training device for a magnetoencephalography system, a traceable positioning method for a magnetoencephalography system, an electronic device, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] Magnetoencephalography (MEG) is a brain function detection technology that non-invasively detects the magnetic field generated by the electrical activity of the brain, has millisecond-level temporal resolution and millimeter-level spatial resolution, and the signal is not affected by tissue conductivity and skull thickness, etc. It can be used for precise positioning of electrical activity in the cerebral cortex, i.e. magnetoencephalography traceable positioning, thereby achieving the functions of assisting brain tumor resection surgery and epilepsy treatment.

[0003] Magnetoencephalography traceable positioning is physically based on the propagation model of magnetoencephalography signals generated by brain electrical activity. The magnetoencephalography signals collected by the magnetoencephalography sensor array are used for inverse solution to determine the source location and intensity of the electrical activity in the cerebral cortex.

[0004] The magnetoencephalography signals measured by magnetoencephalography decay exponentially with distance, and the low signal-to-noise ratio of the magnetoencephalography signals makes it challenging to locate the source in the deep brain region. Since the cerebral cortex is closer to the sensors of the magnetoencephalography system, relatively higher signal-to-noise ratio signals can be obtained. In the related art, the source estimation method assumes that most of the activities recorded by MEG come from the cerebral cortex, so the source space is limited to the surface of the cerebral cortex, and the deep brain region is rarely involved. However, the deep brain region, including the hippocampus and hypothalamus, plays an important role in the function of the brain, involving movement, emotion, memory, and physiological regulation, etc. Studying the magnetoencephalography signals of the deep brain region not only contributes to the progress of basic neuroscience, but also provides important support for clinical applications. Therefore, how to achieve source positioning in the deep brain region is a key problem. SUMMARY

[0005] According to one aspect of the present disclosure, a model training method for a magnetoencephalography system is provided, wherein the magnetoencephalography system comprises a magnetoencephalography helmet, a sensor array fixed on the magnetoencephalography helmet, the magnetoencephalography helmet is located on the head of a subject, and the method comprises: obtaining a three-dimensional model of the brain of the subject and a relative position relationship between the brain and the sensor array; performing mesh division on a preset region of interest in the three-dimensional model to obtain a plurality of simulation source points; generating at least one set of simulation source signals at the plurality of simulation source points in the three-dimensional model; determining at least one set of simulation sensor signals expected to be collected by the sensor array based on the at least one set of simulation source signals using a forward model, wherein the forward model is a mathematical model representing the relationship between the electromagnetic activity of the source points in the brain and the magnetoencephalography signals collected by the sensor array, and the forward model is determined based on the relative position relationship; and training a deep learning model to construct a magnetoencephalography source tracing model by inputting the at least one set of simulation sensor signals and labeling the at least one set of simulation source signals.

[0006] According to another aspect of the present disclosure, a source positioning method for a magnetoencephalography system is provided, wherein the magnetoencephalography system comprises a magnetoencephalography helmet, a sensor array fixed on the magnetoencephalography helmet, the magnetoencephalography helmet is located on the head of a subject to be measured, and the method comprises: obtaining a three-dimensional model of the brain of the subject to be measured; obtaining sensor signals collected by the sensor array; inputting the sensor signals into a magnetoencephalography source tracing model according to the present disclosure to obtain source signals output by the magnetoencephalography source tracing model; and mapping the source signals to a preset region of interest in the three-dimensional model of the brain of the subject to be measured to determine the position information in the source signals.

[0007] According to another aspect of the present disclosure, a model training apparatus for a magnetoencephalography system is provided, wherein the magnetoencephalography system comprises a magnetoencephalography helmet, a sensor array fixed on the magnetoencephalography helmet, the magnetoencephalography helmet is located on a head of a subject, and the apparatus comprises: an acquisition module configured to acquire a three-dimensional model of a brain of the subject and a relative position relationship between the brain and the sensor array; a division module configured to perform mesh division on a preset region of interest in the three-dimensional model to obtain a plurality of simulation source points; a generation module configured to generate at least one set of simulation source signals at the plurality of simulation source points in the three-dimensional model; a determination module configured to determine, based on the at least one set of simulation source signals, at least one set of simulation sensor signals expected to be collected by the sensor array using a forward model, wherein the forward model is a mathematical model representing a relationship between electromagnetic activity of a source point in the brain and magnetoencephalography signals collected by the sensor array, and the forward model is determined based on the relative position relationship; and a training module configured to train a deep learning model to construct a magnetoencephalography source tracing model with the at least one set of simulation sensor signals as input and the at least one set of simulation source signals as labels.

[0008] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores a computer program which, when executed by the at least one processor, implements the model training method according to the present disclosure.

[0009] According to another aspect of the present disclosure, one or more computer readable storage media are provided, having instructions stored thereon, which, in response to being executed by one or more processors, cause the one or more processors to perform the model training method according to the present disclosure.

[0010] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements the model training method according to the present disclosure.

[0011] In the present disclosure, a simulation source signal emitted by the brain is simulated through a constructed three-dimensional model, and a simulated sensor signal collected by a sensor array is estimated through a forward model, so as to form a training set for training a deep learning model. The trained brain magnetic source localization model can be used for brain magnetic source localization, which has strong robustness in noise suppression, can reduce the influence of signals with low signal-to-noise ratio detected by the sensor on source localization positioning, thereby improving the positioning accuracy of source localization, and is suitable for source localization of deep brain regions. In addition, the trained brain magnetic source localization model is an end-to-end model, which takes sensor signals in the sensor space as input and source signals in the brain space as output, can directly utilize signals collected by sensors of a magnetoencephalogram, and generate source signals that can be directly mapped to the brain space without any other processing.

[0012] These and other aspects of the present disclosure will become clear from the embodiments described hereinafter and will be apparent from the embodiments described hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0013] In the following description of the example embodiments in conjunction with the accompanying drawings, more details, features and advantages of the present disclosure are disclosed, in which:

[0014] Figure 1 A flowchart of a model training method for a magnetoencephalography system according to some example embodiments of the present disclosure is shown;

[0015] Figure 2 A flowchart of a step of acquiring a three-dimensional model and a relative position relationship in the model training method of Figure 1

[0016] Figure 3 A flowchart of a source localization positioning method for a magnetoencephalography system according to some example embodiments of the present disclosure is shown;

[0017] Figure 4 A schematic block diagram of a model training device for a magnetoencephalography system according to an example embodiment of the present disclosure is shown;

[0018] Figure 5 A schematic block diagram of a source localization positioning device for a magnetoencephalography system according to an example embodiment of the present disclosure is shown; and

[0019] Figure 6 An example configuration of an electronic device that can be used to implement the methods described herein is shown. DETAILED DESCRIPTION

[0020] ​In the following, only certain exemplary embodiments are simply described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the present application. Therefore, the drawings and description are considered to be exemplary in nature rather than limiting.

[0021] Source localization of magnetoencephalography is an ill-posed inverse problem, that is, estimating brain activity from a small number of sensors, possibly thousands of locations, an infinite number of source activity patterns can appear as a sensor signal distribution. With the rapid development of deep learning models, it provides state-of-the-art results for X-ray computed tomography, magnetic resonance image reconstruction, natural image restoration, etc., and shows strong robustness in noise suppression. Therefore, the deep learning model can be used to process the ill-posed inverse problem of magnetoencephalography source localization.

[0022] In the present disclosure, the simulated source signal emitted by the brain is simulated by the constructed three-dimensional model, and the sensor signal collected by the sensor array is estimated by the forward model, thereby forming a training set to train the deep learning model. The trained magnetoencephalography source localization model can be used for magnetoencephalography source localization, which shows strong robustness in noise suppression, can reduce the influence of signals with low signal-to-noise ratio detected by sensors on source localization, thereby improving the accuracy of source localization, and is suitable for source localization of deep brain regions. In addition, the trained magnetoencephalography source localization model is an end-to-end model, which takes the sensor signal in the sensor space as the input and the source signal in the brain space as the output, can directly use the signal collected by the sensor of magnetoencephalography, and generate the source signal that can be directly mapped to the brain space without any other processing.

[0023] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0024] Figure 1A flowchart of a model training method 1000 for a magnetoencephalography system is shown according to some example embodiments of the present disclosure. The magnetoencephalography system can include a magnetoencephalography helmet, a sensor array fixed on the magnetoencephalography helmet, and the magnetoencephalography helmet is located on a head of a subject. The sensor array is used to collect a magnetoencephalography signal of a brain (also referred to as a cerebrum, located in the head) of a subject to be measured. The brain of a human body can generate electromagnetic activity at one or more source field points (i.e., a certain position point in the brain) when excited, which can be detected by the sensor array to output a magnetoencephalography signal. In a magnetoencephalography source localization process, the position information of the source field point in the brain that generates the electromagnetic activity (i.e., related to the magnetoencephalography signal) can be obtained by performing reverse solving based on the magnetoencephalography signal collected by the sensor array and a mapping relationship (e.g., a forward model, which is a mathematical model describing the relationship between the electromagnetic activity of the source point in the brain and the magnetoencephalography signal collected by the sensor array) between a source space of the brain of the subject to be measured and a sensor space of the sensor array.

[0025] As shown in Figure 1 The model training method 1000 includes: obtaining a three-dimensional model of the brain of the subject and a relative position relationship between the brain and the sensor array, S101; performing mesh division on a preset region of interest in the three-dimensional model to obtain a plurality of simulated source points, S102; generating at least one set of simulated source signals at the plurality of simulated source points in the three-dimensional model, S103; determining at least one set of simulated sensor signals expected to be collected by the sensor array based on the at least one set of simulated source signals using a forward model, S104, wherein the forward model is a mathematical model representing the relationship between the electromagnetic activity of the source point in the brain and the magnetoencephalography signal collected by the sensor array, and the forward model is determined based on the relative position relationship; and training a deep learning model to construct a magnetoencephalography source localization model by taking the at least one set of simulated sensor signals as input and the at least one set of simulated source signals as labels, S105.

[0026] The magnetoencephalography source localization model formed in the above manner can be used for magnetoencephalography source localization. For example, the sensor signal collected by the sensor array is input into the magnetoencephalography source localization model to obtain a source signal corresponding to the sensor signal output by the model, and then the source signal is mapped into an anatomical structure model of the brain to observe the distribution of the source signal, thereby obtaining the position information of the source localization. The magnetoencephalography source localization model has strong robustness in noise suppression due to the use of the deep learning model, which can reduce the influence of the signal with low signal-to-noise ratio detected by the sensor on the source localization, thereby improving the accuracy of the source localization, and is suitable for source localization of deep brain regions.

[0027] In some embodiments, the three-dimensional model of the brain is an anatomical model of the brain, which can be formed using images of the anatomy of the head acquired by a medical imaging device (e.g., a nuclear magnetic resonance device, a computed tomography device, etc.).

[0028] In some embodiments, the relative positional relationship between the brain and the sensor array can include a relative positional relationship between a preset region of interest in the brain and the sensor array. The preset region of interest can include the whole brain, the hippocampus, one of the thalamus, or other deep brain regions. In the case where the preset region of interest includes the whole brain, the complexity of model training can be increased, but the above-mentioned method of constructing a training set (i.e., generating a simulated source signal and a simulated sensor signal) and training a deep learning model according to the present disclosure can alleviate the above-mentioned problem.

[0029] In some embodiments, as shown in FIG. 1, the step S101 of acquiring a three-dimensional model of the brain of the subject and a relative positional relationship between the brain and the sensor array can include the following steps: Figure 2 In some embodiments, as shown in FIG. 1, the step S101 of acquiring a three-dimensional model of the brain of the subject and a relative positional relationship between the brain and the sensor array can include the following steps:

[0030] In some embodiments, the at least one set of simulated source signals can comprise a plurality of sets of simulated source signals, and the step S103 of generating the at least one set of simulated source signals at the plurality of simulated source points in the three-dimensional model can comprise: performing the step of generating a set of simulated source signals at the plurality of simulated source points in the three-dimensional model for a plurality of times to obtain the plurality of sets of simulated source signals. Each set of simulated source signals (or in other words, each of the plurality of sets of simulated source signals) can comprise a simulated source signal at each of the plurality of simulated source points. In order to simulate the real electromagnetic activity of each of the source points in the brain, the plurality of simulated source points can be activated at least partially in each of the plurality of times, and each of the plurality of simulated source points can simulate a simulated source signal. The simulated source signals can be strong signals, weak signals, or even weak signals to zero. Alternatively, each set of simulated source signals can comprise simulated source signals at a portion of the plurality of simulated source points. By performing the step of generating a set of simulated source signals at the plurality of simulated source points in the three-dimensional model for a plurality of times, the plurality of sets of simulated source signals can be obtained, so as to train the deep learning model more effectively.

[0031] In some embodiments, the step of performing multiple times to generate a set of simulated source signals at the plurality of simulated source points in the three-dimensional model can include: in each time of performing the step of generating simulated source signals, applying a simulated signal to at least one simulated source point in the plurality of simulated source points as a seed source to obtain the simulated source signals at each simulated source point in the plurality of simulated source points in this time. Wherein the signals generated at the simulated source points around each simulated source point as a seed source attenuate according to a Gaussian distribution as the distance to the simulated source point increases, and the simulated signal has a time dimension. In order to simulate the real electromagnetic activity of the brain, a region growing method can be used to simulate a plurality of source points generating source signals and the generated source signals in a preset region of interest of the three-dimensional model. In other words, in each simulation, at least one source cluster is formed from the plurality of simulated source points in the three-dimensional model using the region growing method. Each source cluster includes a simulated source point as a seed source and its surrounding simulated source points meeting certain conditions. The simulated source points included in each source cluster can be completely different, or partially the same and partially different (at this time, the simulated source signals of the simulated source points included in multiple source clusters at the same time can be the superposition of the signals assigned by multiple source clusters). Specifically, a simulated signal is applied to at least one simulated source point in the plurality of simulated source points as a seed source to form at least one source cluster, each source cluster includes a seed source (one seed source can be one simulated source point, or multiple simulated source points), and the signals generated at the simulated source points in the source cluster attenuate according to a Gaussian distribution as the distance to the seed source (i.e., the simulated source point as a seed source) included in the source cluster increases, thereby simulating the source points generating electromagnetic activity and the generated signals in the three-dimensional model. The specific way of performing the above steps can be, for example: each simulation forms at least one source cluster by the region growing method, the signals generated at the simulated source points in the source cluster attenuate according to a Gaussian distribution as the distance to the seed source increases (each source cluster can have one simulated source point as a seed source, or two or more simulated source points as seed sources), thereby generating a plurality of sets of data; adding a time dimension to the generated plurality of sets of data, and the activation signal of the seed source of the source cluster is a simulated signal containing multiple sampling points. Alternatively, other ways can also be used to simulate the simulated source signals generated at the plurality of simulated source points in the three-dimensional model.

[0032] In some embodiments, the simulated signal includes at least one of a sinusoidal signal and a central peak simulated signal.

[0033] In some embodiments, step S104, determining, based on the at least one set of simulated source signals, the at least one set of simulated sensor signals expected to be collected by the sensor array using the forward model, can include: determining, based on each set of simulated source signals in the plurality of sets of simulated source signals, a set of simulated sensor signals in the plurality of sets of simulated sensor signals corresponding to the set of simulated source signals using the forward model. That is, the at least one set of simulated source signals in the brain space can be projected into the sensor space through the forward model to estimate the simulated sensor signals corresponding to the at least one set of simulated source signals. Specifically, for example, for each set of simulated source signals J, a set of simulated sensor signals Sm=KJ in the plurality of sets of simulated sensor signals corresponding to the set of simulated source signals J is determined using the forward model K. In this way, the sensor signals corresponding to the simulated source signals that can be collected by the sensor array can be more accurately estimated.

[0034] In some embodiments, S105, training the deep learning model to construct the magnetoencephalography source localization model with the at least one set of simulated sensor signals as input and the at least one set of simulated source signals as label, can include: for each set of simulated source signals in the plurality of sets of simulated source signals, training the deep learning model to construct the magnetoencephalography source localization model with a set of simulated sensor signals corresponding to the set of simulated source signals as input and the set of simulated source signals as label (i.e., output). That is, the plurality of sets of simulated source signals and the plurality of sets of simulated sensor signals exist in one-to-one correspondence pairs. When training, each time a set of simulated sensor signals corresponding to a set of simulated source signals is input, and the set of simulated source signals is labeled, the deep learning model is trained, thereby facilitating effective training of the deep learning model. The set of simulated sensor signals for training can be MxY dimensions, where M is the number of sensors in the sensor array, and Y is the number of sampling points in the simulated signal. In some other examples, the set of simulated sensor signals can also take other dimensions as input according to actual conditions. The set of simulated source signals for training can be QxY dimensions, where Q is the number of simulated source points obtained by grid division of the preset region of interest.

[0035] In some embodiments, the model training method 1000 can further include adjusting each of the at least one set of simulated sensor signals using a noise signal. At this time, training the deep learning model to construct the magnetoencephalography source tracing model with the at least one set of simulated sensor signals as input and the at least one set of simulated source signals as labels includes training the deep learning model to construct the magnetoencephalography source tracing model with the adjusted set of simulated sensor signals corresponding to each of the at least one set of simulated source signals as input and the set of simulated source signals as labels. That is, noise can be added to the simulated sensor signals to simulate relatively real sensor signals. For example, the adjusted set of simulated sensor signals Sm’ = Sm + n(t) (where Sm is the set of simulated sensor signals Sm = KJ corresponding to a set of simulated source signals J determined using the forward model K, and n(t) is a noise signal n(t)). At this time, the deep learning model is trained with the adjusted set of simulated sensor signals Sm’ corresponding to the set of simulated source signals J as input and the set of simulated source signals J as labels.

[0036] In some embodiments, the noise signal includes at least one of Gaussian white noise with different signal-to-noise ratios, signal noise collected by a real sensor.

[0037] In some embodiments, the deep learning model includes a convolutional neural network (cnn), a deep residual network (ResNet), a long short-term memory network (LSTM), a VGGNet, a ViT model, etc.

[0038] In the above implementation, the training data set is obtained in a computer simulation manner, so as to ensure that the data in the training data set is relatively accurate, thereby improving the accuracy of the trained magnetoencephalography source tracing model.

[0039] According to one or more of the above embodiments, taking the hippocampus as an example, the model training method can be, for example: obtaining a three-dimensional model of the brain of the subject and the relative position relationship between the hippocampus and the sensor array (including 64 sensors); performing grid division on the hippocampus in the three-dimensional model to obtain 1000 simulation source points; performing the step of generating a set of simulation source signals at the 1000 simulation source points in the three-dimensional model multiple times to obtain multiple sets of simulation source signals, for example, the specific manner of performing the above step each time can be that each time simulation forms at least one source cluster (each source cluster can have one simulation source point as a seed source, or more than two simulation source points as seed sources, the source cluster is generated by a region growing method, and the signal generated at the simulation source points in the source cluster is distributed according to a Gaussian distribution with the increase of the distance from the seed source Attenuation extreme distance), thereby generating 100000 sets of data, then adding a time dimension to the generated data (the activation signal of the seed source of the source cluster is a simulation signal containing 100 sampling points), to generate a set of simulation source signals J; for each set of simulation source signals J, using the forward model K to determine a set of simulation sensor signals Sm=KJ expected to be collected by the 64 sensors; adjusting each set of simulation sensor signals in at least one set of simulation sensor signals using a noise signal, that is, adding a noise signal to each set of simulation sensor signals Sm to generate an adjusted each set of simulation sensor signals Sm’=KJ+n(t); for each set of simulation source signals J in the multiple sets of simulation source signals, using the adjusted set of simulation sensor signals Sm’ corresponding to the set of simulation source signals J as input and the set of simulation source signals J as a label, training a deep learning model to construct a magnetoencephalography source tracing model, each set of simulation sensor signals is 64x100 dimensions, and each set of simulation source signals is 1000x100 dimensions. The deep learning model uses a deep residual network (ResNet). The above trained magnetoencephalography source tracing model can predict the data collected by the sensor to obtain the source signal associated therewith, that is, output 1000x100 dimensional source signals. Then the 1000x100 dimensional source signals can be mapped to the anatomical structure of the hippocampus, and the distribution of the source signals is observed to perform source positioning.

[0040] For example, the model training method can be as follows, taking the thalamus as a preset region of interest: a three-dimensional model of the subject's brain and the relative position relationship between the thalamus and the sensor array (including 128 sensors) are obtained; the thalamus in the three-dimensional model is meshed to obtain 2000 simulation source points; the step of generating a set of simulation source signals at the 2000 simulation source points in the three-dimensional model is performed multiple times to obtain multiple sets of simulation source signals, for example, the specific way of performing the above step each time can be that each time simulation forms at least one source cluster (each source cluster can have one simulation source point as a seed source, or more than two simulation source points as seed sources, the source cluster is generated by a region growing method, and the signal generated at the simulation source points in the source cluster is attenuated according to a Gaussian distribution with the increase of the distance from the seed source), thereby generating 100,000 sets of data, then, the generated data is added with a time dimension (the activation signal of the seed source of the source cluster is a simulation signal containing 1000 sampling points), to generate a set of simulation source signals J; for each set of simulation source signals J, a forward model K is used to determine a set of simulation sensor signals Sm=KJ expected to be collected by the 128 sensors; each set of simulation sensor signals is adjusted using a noise signal, that is, a noise signal is added to each set of simulation sensor signals Sm to generate an adjusted set of simulation sensor signals Sm'=KJ+n(t); for each set of simulation source signals J in the multiple sets of simulation source signals, a deep learning model is trained to construct a magnetoencephalography (MEG) source tracing model, taking the adjusted set of simulation sensor signals Sm' corresponding to the set of simulation source signals J as input and the set of simulation source signals J as a label, each set of simulation sensor signals is 128x1000-dimensional, and each set of simulation source signals is 2000x1000-dimensional. The deep learning model uses a long short-term memory network (LSTM). The MEG source tracing model trained above can predict the data collected by the sensors to obtain the source signals associated therewith, that is, output 2000x1000-dimensional source signals. Then, the 2000x1000-dimensional source signals can be mapped to the anatomical structure of the thalamus, and the distribution of the source signals is observed to perform source localization.

[0041] For example, the preset region of interest is the whole brain, the model training method can be: obtaining a three-dimensional model of the brain of the subject and the relative position relationship between the whole brain and the sensor array (including 256 sensors); performing grid division on the whole brain in the three-dimensional model to obtain 2500 simulation source points; performing the step of generating a set of simulation source signals at the 2500 simulation source points in the three-dimensional model multiple times to obtain multiple sets of simulation source signals, for example, the specific way of performing the above step each time can be: each time simulation forms at least one source cluster (each source cluster can have one simulation source point as a seed source, or more than two simulation source points as seed sources, the source cluster is generated by a region growing method, and the signal generated at the simulation source points in the source cluster is distributed according to a Gaussian distribution with the increase of the distance from the seed source Attenuation extreme distance), thereby generating 500,000 sets of data, then adding a time dimension to the generated data (the activation signal of the seed source of the source cluster is a simulation signal containing 1000 sampling points), to generate a set of simulation source signals J; for each set of simulation source signals J, using the forward model K to determine a set of simulation sensor signals Sm=KJ expected to be collected by the 256 sensors; adjusting each set of simulation sensor signals using a noise signal, that is, adding a noise signal to each set of simulation sensor signals Sm to generate an adjusted each set of simulation sensor signals Sm'=KJ+n(t); for each set of simulation source signals J in the multiple sets of simulation source signals, using the adjusted set of simulation sensor signals Sm' corresponding to the set of simulation source signals J as input and the set of simulation source signals J as label, training a deep learning model to construct a magnetoencephalography source localization model, each set of simulation sensor signals is 256x1000 dimensions, and each set of simulation source signals is 2500x1000 dimensions. The deep learning model uses a deep residual network (ResNet). The above trained magnetoencephalography source localization model can predict the data collected by the sensor to obtain the source signal associated therewith, that is, output 2500x1000 dimensional source signals. Then the 2500x1000 dimensional source signals can be mapped to the anatomical structure of the whole brain, and the distribution of the source signals is observed to perform source localization.

[0042] Figure 3 A flowchart of a source localization method 3000 for a magnetoencephalography system is shown according to some example embodiments of the present disclosure. The magnetoencephalography system includes a magnetoencephalography helmet, a sensor array fixed on the magnetoencephalography helmet, and the magnetoencephalography helmet is located on the head of a subject to be measured. As shown in FIG. 30, the source localization method 3000 includes the following steps: Figure 3As shown, the source localization method 3000 comprises: step S301, obtaining a three-dimensional model of the brain of the to-be-tested subject; step S302, obtaining a sensor signal collected by a sensor array; step S303, inputting the sensor signal into the magnetoencephalography source localization model constructed according to the model training method 1000 to obtain a source signal output by the magnetoencephalography source localization model; and step S304, mapping the source signal into a preset region of interest of the three-dimensional model of the brain of the to-be-tested subject to determine the position information in the source signal. The method for obtaining the three-dimensional model of the brain of the to-be-tested subject in step S301 is the same as the method for obtaining the three-dimensional model of the brain of the subject in the model training method 1000, and will not be described in detail herein. In addition, the preset region of interest in the model training method 1000 is the same region as the preset region of interest in the source localization method 3000. That is, when the preset region of interest to which the model training method 1000 is directed is, for example, the hippocampus, the constructed magnetoencephalography source localization model is also directed to the hippocampus, and the source localization method implemented by the magnetoencephalography source localization model is also directed to the source localization of the hippocampus.

[0043] The source localization method described above adopts the source localization model of the present disclosure, and thus has strong robustness in noise suppression, can reduce the influence of low signal-to-noise ratio of the signal detected by the sensor on the source localization, thereby improving the accuracy of the source localization, and is suitable for source localization of deep brain regions.

[0044] Figure 4 A schematic block diagram of a model training apparatus 4000 for a magnetoencephalography system according to an exemplary embodiment of the present disclosure is shown. As shown in the figure, Figure 4 The model training apparatus 4000 can comprise an obtaining module 401, a dividing module 402, a generating module 403, a determining module 404, and a training module 405. The obtaining module 401 is configured to obtain a three-dimensional model of the brain of a subject and a relative position relationship between the brain and a sensor array. The dividing module 402 is configured to perform grid division on a preset region of interest in the three-dimensional model to obtain a plurality of simulated source points. The generating module 403 is configured to generate at least one set of simulated source signals at the plurality of simulated source points in the three-dimensional model. The determining module 404 is configured to determine at least one simulated sensor signal expected to be collected by the sensor array based on the at least one set of simulated source signals using a forward model, wherein the forward model is a mathematical model representing the relationship between the electromagnetic activity of the source points in the brain and the magnetoencephalography signals collected by the sensor array, and the forward model is determined based on the relative position relationship. The training module 405 is configured to train a deep learning model to construct a magnetoencephalography source localization model by taking the at least one set of simulated sensor signals as input and the at least one set of simulated source signals as label.

[0045] It should be understood that Figure 4 The modules of the model training apparatus 4000 shown in the figure can be the same as the modules of the model training method 1000 described with reference to Figure 1The various steps in the described method 1000 correspond. Thus, the operations, features and advantages described above for the method 1000 apply equally to the model training apparatus 4000 and the modules comprised therein. For the sake of brevity, certain operations, features and advantages are not repeated here.

[0046] Figure 5 A schematic block diagram of a source localization apparatus 5000 for magnetoencephalography system according to an example embodiment of the present disclosure is shown. As Figure 5 shown, the source localization apparatus 5000 can comprise a first obtaining module 501, a second obtaining module 502, an inputting module 503, a mapping module 504. The first obtaining module 501 is configured to obtain a three-dimensional model of a brain of a subject to be measured. The second obtaining module 502 is configured to obtain sensor signals acquired by a sensor array. The inputting module 503 is configured to input the sensor signals into a magnetoencephalography source model to obtain source signals output by the magnetoencephalography source model. The mapping module 504 is configured to map the source signals into a pre-set region of interest of the three-dimensional model of the brain of the subject to be measured to determine location information in the source signals.

[0047] It should be understood that Figure 5 the various modules of the source localization apparatus 5000 shown in FIG. 5 can correspond to the modules of the model training apparatus 4000 shown in FIG. 4. Figure 3 The various steps in the described method 3000 correspond. Thus, the operations, features and advantages described above for the method 3000 apply equally to the source localization apparatus 5000 and the modules comprised therein. For the sake of brevity, certain operations, features and advantages are not repeated here.

[0048] According to yet another aspect of the present disclosure, there is provided an electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the processor is configured to execute the computer program to implement the above-described method 1000 and / or 3000.

[0049] According to yet another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the above-described method 1000 and / or 3000.

[0050] According to yet another aspect of the present disclosure, there is provided a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the above-described method 1000 and / or 3000.

[0051] In the following, illustrative examples of such computer devices, non-transitory computer readable storage media and computer program products are described. Figure 6

[0052] Figure 6 ​An example configuration of an electronic device 6000 that can be used to implement the modules and functionality described herein is shown.

[0053] The electronic device 6000 can be various different types of devices such as a server of a service provider, a device associated with a client (e.g., a client device), a system on chip, and / or any other suitable electronic device or computing system. Examples of the electronic device 6000 include, but are not limited to: a desktop computer, a server computer, a notebook or netbook computer, a mobile device (e.g., a tablet or phablet device, a cellular or other wireless phone (e.g., a smart phone), a notepad computer, a mobile station), a wearable device (e.g., glasses, a watch), an entertainment device (e.g., an entertainment appliance, a set-top box, a game console), a television or other display device, an automobile computer, and so forth. Thus, the electronic device 6000 can range from full resource devices with substantial memory and processor resources (e.g., personal computers, game consoles) to a low-resource device with limited memory and / or processing resources (e.g., a traditional set-top box, a handheld game console).

[0054] The electronic device 6000 can include at least one processor 602, memory 604, communication interface(s) 606, a display device 608, other input / output (I / O) devices 610, and one or more mass storage devices 612, which each can communicate with one another by way of a system bus 614 or other appropriate communication link.

[0055] The processor 602 can be a single processing unit or a plurality of processing units, all of which can include single or multiple computing units or multiple cores. The processor 602 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and / or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor 602 can be configured to fetch and execute computer-readable instructions stored in the memory 604, the mass storage device 612, or any other computer-readable medium.

[0056] Memory 604 and mass storage device 612 are examples of computer storage media for storing instructions that are executed by processor 602 to implement the various functions described above. By way of example, the memory 604 can generally include both volatile memory and nonvolatile memory (e.g., RAM, ROM, etc.). In addition, mass storage device 612 can generally include hard disk drives, solid state drives, removable media, including external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CD, DVD), storage arrays, network attached storage, storage area networks, etc. Both memory 604 and mass storage device 612 can be collectively referred to herein as memory or computer storage media, and can be non-transitory media capable of storing computer-readable, processor-executable program instructions as computer program code that can be executed by processor 602 as a particular machine configured to implement the operations and functions described in the examples herein.

[0057] A number of program modules can be stored on mass storage device 612. These programs include operating system 616, one or more application programs 618, other programs 620, and program data 622, and they can be loaded into memory 604 for execution. Examples of such application programs or program modules can include, for example, computer program logic (e.g., computer program code or instructions) for implementing the model training apparatus 4000 (acquisition module 401, partitioning module 402, generation module 403, determination module 404, training module 405), the provenance localization apparatus 5000 (first acquisition module 501, second acquisition module 502, input module 503, mapping module 504), the method 1000 (including any suitable steps of the method 1000), the method 3000 (including any suitable steps of the method 3000), and / or additional embodiments described herein, for example.

[0058] Although illustrated in Figure 6 as being stored in memory 604 of electronic device 6000, modules 616, 618, 620, and 622, or portions thereof, can be implemented using any form of computer-readable media that is accessible by electronic device 6000. As used herein, "computer-readable media" includes both computer storage media and communication media.

[0059] Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by an electronic device.

[0060] In contrast, communication media can embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism. As defined herein, computer storage media does not include communication media.

[0061] The electronic device 6000 can also include one or more communication interfaces 606 for exchanging data with other devices, such as over a network, direct connection, etc., as previously discussed. Such communication interfaces can be one or more of: any type of network interface (e.g., network interface card (NIC)), wired or wireless (such as IEEE 802.11 wireless LAN (WLAN)) wireless interface, Worldwide Interoperability for Microwave Access (Wi-MAX) interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth™ interface, near field communication (NFC) interface, etc. The communication interfaces 606 can facilitate communications within a variety of networks and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc. The communication interfaces 606 can also provide communication with external storage devices (not shown), such as storage arrays, network attached storage, storage area networks, etc.

[0062] In some examples, a display device 608, such as a monitor, can be included for displaying information and images to a user. Other I / O devices 610 can be devices that receive various inputs from a user and provide various outputs to the user, and can include touch input devices, gesture input devices, cameras, keyboards, remote controls, mice, printers, audio input / output devices, etc.

[0063] While the disclosure has been illustrated and described in detail in the drawings and foregoing description, such illustration and description is to be considered illustrative or exemplary and not restrictive; the disclosure is not limited to the disclosed embodiments. Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed subject matter, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps not listed in the claims, the indefinite article "a" or "an" does not exclude a plurality, and the term "plurality" means two or more. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. A model training method for a magnetoencephalography (MEG) system, wherein, The magnetoencephalography (MEG) system includes a MEG helmet and a sensor array fixed to the MEG helmet, the MEG helmet being positioned on the subject's head, and the method includes: Obtain a three-dimensional model of the subject's brain and the relative positional relationship between the brain and the sensor array; The preset regions of interest in the three-dimensional model are divided into meshes to obtain multiple simulated source points, wherein the preset regions of interest include deep brain regions; At least one set of analog source signals is generated at multiple analog source points in the three-dimensional model; Based on the at least one set of simulated source signals, a forward model is used to determine at least one set of simulated sensor signals that the sensor array is expected to acquire, wherein the forward model is a mathematical model representing the relationship between the electromagnetic activity of source points in the brain and the brain magnetic signals acquired by the sensor array, and the forward model is determined based on the relative positional relationship; and Using the at least one set of simulated sensor signals as input and the at least one set of simulated source signals as labels, a deep learning model is trained to construct a magnetoencephalography (MEG) source tracing model. Wherein, at least one set of simulated source signals includes multiple sets of simulated source signals, wherein each set of simulated source signals includes a simulated source signal at each of the multiple simulated source points, and the step of generating a set of simulated source signals at the multiple simulated source points in the three-dimensional model is performed multiple times to obtain the multiple sets of simulated source signals, including: In each step of generating a simulated source signal, a simulated signal is applied with at least one of the plurality of simulated source points as a seed source to obtain a simulated source signal at each of the plurality of simulated source points, wherein the signal generated at the simulated source points around each simulated source point serving as the seed source decays according to a Gaussian distribution as the distance from the simulated source point increases, and the simulated signal has a time dimension.

2. The method according to claim 1, wherein, Generating at least one set of analog source signals at multiple analog source points in the three-dimensional model includes: The step of generating a set of simulated source signals at multiple simulated source points in the three-dimensional model is performed multiple times to obtain the multiple sets of simulated source signals.

3. The method according to claim 2, wherein, The at least one set of analog sensor signals includes multiple sets of analog sensor signals, and wherein, Based on the at least one set of analog source signals, the at least one set of analog sensor signals that the sensor array is expected to acquire, determined using a forward model, includes: Based on each set of analog source signals from the plurality of sets of analog source signals, a forward model is used to determine a set of analog sensor signals from the plurality of sets of analog sensor signals that corresponds to that set of analog source signals, and Training a deep learning model to construct a magnetoencephalography (MEG) source tracing model by using the at least one set of analog sensor signals as input and the at least one set of analog source signals as labels includes: For each of the multiple sets of simulated source signals, the deep learning model is trained using a set of simulated sensor signals corresponding to that set of simulated source signals as input and the set of simulated source signals as labels to construct a magnetoencephalography (MEG) source tracing model.

4. The method according to claim 1, wherein, The analog signal includes at least one of a sinusoidal signal and a center-peak analog signal.

5. The method according to any one of claims 1 to 4, wherein, The method further includes: adjusting each set of analog sensor signals in the at least one set of analog sensor signals using noise signals, and training a deep learning model to construct a magnetoencephalography (MEG) source tracing model using the at least one set of analog sensor signals as input and the at least one set of analog source signals as labels. For each of the at least one set of simulated source signals, a set of adjusted simulated sensor signals corresponding to that set of simulated source signals is used as input and the set of simulated source signals is used as labels to train a deep learning model to construct a magnetoencephalography (MEG) source tracing model.

6. The method according to claim 5, wherein, The noise signal includes at least one of Gaussian white noise with different signal-to-noise ratios and signal noise acquired by a real sensor.

7. The method according to any one of claims 1 to 4, wherein, Obtaining a three-dimensional model of the subject's brain and the relative positional relationship between the brain and the sensor array includes: Anatomical images of the subject's head were acquired using medical imaging equipment; The anatomical images of the head are used to segment and reconstruct the brain to generate a three-dimensional model of the brain; and The scalp surface of the head is registered with the position of the sensor array to obtain the relative positional relationship between the subject's brain and the sensor array.

8. The method according to claim 7, wherein, The relative positional relationship between the brain and the sensor array includes the relative positional relationship between a preset region of interest in the brain and the sensor array, and the preset region of interest includes one of the whole brain, the hippocampus, and the thalamus.

9. A method for tracing and locating the source of an event in a magnetoencephalogram (MEG) system, wherein, The magnetoencephalography (MEG) system includes a MEG helmet and a sensor array fixed on the MEG helmet, the MEG helmet being positioned on the head of the subject being tested, and the method includes: Obtain a three-dimensional model of the brain of the subject under test; Acquire the sensor signals collected by the sensor array; The sensor signal is input into the magnetoencephalography (MEG) source tracing model according to any one of claims 1 to 8 to obtain the source signal output by the MEG source tracing model; and The source signal is mapped onto a preset region of interest in a three-dimensional model of the brain of the subject to be tested, in order to determine the location information in the source signal.

10. A model training device for a magnetoencephalography (MEG) system, wherein, The magnetoencephalography (MEG) system includes a MEG helmet and a sensor array fixed to the MEG helmet, the MEG helmet being positioned on the subject's head, and the device includes: An acquisition module is configured to acquire a three-dimensional model of the subject's brain and the relative positional relationship between the brain and the sensor array; A partitioning module is configured to partition a preset region of interest in the 3D model into a mesh to obtain multiple simulated source points, wherein the preset region of interest includes the deep brain. A generation module, configured to generate at least one set of analog source signals at multiple analog source points in the three-dimensional model; A determining module, configured to determine, based on the at least one set of analog source signals, at least one set of analog sensor signals that the sensor array is expected to acquire, using a forward model, wherein the forward model is a mathematical model representing the relationship between the electromagnetic activity of source points in the brain and the magnetoencephalogram (MEG) signals acquired by the sensor array, and the forward model is determined based on the relative positional relationship; and A training module, configured to train a deep learning model to construct a magnetoencephalography (MEG) source tracing model using the at least one set of analog sensor signals as input and the at least one set of analog source signals as labels. Wherein, at least one set of simulated source signals includes multiple sets of simulated source signals, wherein each set of simulated source signals includes a simulated source signal at each of the multiple simulated source points, and the step of generating a set of simulated source signals at the multiple simulated source points in the three-dimensional model is performed multiple times to obtain the multiple sets of simulated source signals, including: In each step of generating a simulated source signal, a simulated signal is applied with at least one of the plurality of simulated source points as a seed source to obtain a simulated source signal at each of the plurality of simulated source points, wherein the signal generated at the simulated source points around each simulated source point serving as the seed source decays according to a Gaussian distribution as the distance from the simulated source point increases, and the simulated signal has a time dimension.

11. An electronic device, the electronic device comprising: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores a computer program that, when executed by the at least one processor, implements the method according to any one of claims 1-8.

12. One or more computer-readable storage media having instructions stored thereon, the instructions being responsive to execution by one or more processors to cause the one or more processors to perform the method of any one of claims 1-8.

13. A computer program product comprising a computer program, wherein, The computer program, when executed by a processor, implements the method according to any one of claims 1-8.

Images