Brain power source imaging method, device, equipment, storage medium and product

Abstract

The invention relates to the technical field of signal processing, and discloses an electroencephalogram imaging method, device and equipment, a storage medium and a product. The multi-channel electroencephalogram signals are preprocessed; inputting the preprocessed multi-channel electroencephalogram signals into a pre-constructed generalized variable grouping sparse constraint source imaging model, and outputting an electroencephalogram source imaging result; the source imaging loss function is solved through a preset deep learning network model in the generalized variable grouping sparse constraint source imaging model. According to the method, a regular term constructed by two kinds of prior knowledge is introduced into a generalized variable grouping sparse constraint source imaging model to serve as a loss function, so that the robustness and accuracy of a brain power source imaging result are guaranteed. Besides, a deep learning model is adopted for solving, and the constraint model and the deep learning model are combined, so that the solving efficiency of the ill-conditioned inverse problem can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of signal processing, and particularly relates to a method, device, equipment, storage medium and product for brain power imaging. Background Art

[0002] Brain power imaging technology is to infer the distribution and dynamic evolution of internal electrical activity sources in the brain by recording electroencephalogram (EEG) signals observed on the scalp, and obtain high-resolution imaging of the dynamic cerebral cortex.

[0003] However, when inferring the internal electrical activity sources in the brain from the observed EEG signals, there are certain mathematical difficulties, and existing algorithms face significant challenges in solving this ill-posed inverse problem. For example: the minimum norm estimation (MNE) based on the l2-norm and its improved algorithms (such as sLORETA), due to their excessive dependence on smoothing constraints, result in limited source space resolution and are prone to spatial diffusion artifacts; while the sparse imaging method using l1-norm regularization, although it can improve the focusing property, is prone to underestimating the true source space range due to excessive sparsification. Moreover, real-time source imaging also faces the problem of computational efficiency bottlenecks. Although methods such as beamformers attempt to optimize the update of the weight matrix, such as the dynamic calculation based on the covariance matrix, the solution of complex inverse problems is still difficult to complete within millisecond-level delays, especially when processing high-density electrode data. These have severely restricted the application of brain power imaging technology in the fields of brain disease diagnosis and brain-computer interfaces. Summary of the Invention

[0004] In view of this, the present invention provides a method, device, equipment, storage medium and product for brain power imaging to solve the problem of low solution efficiency in solving the ill-posed inverse problem in the prior art.

[0005] In a first aspect, the present invention provides a method for brain power imaging, the method comprising:

[0006] Obtain multi-channel electroencephalogram signals;

[0007] Preprocess the multi-channel electroencephalogram signals;

[0008] Input the preprocessed multi-channel electroencephalogram signals into a pre-constructed generalized variable group sparse constraint source imaging model, and output a brain power imaging result; in the generalized variable group sparse constraint source imaging model, a preset deep learning network model is used to solve the source imaging loss function.

[0009] The present invention introduces a regular term constructed by two kinds of prior knowledge as a loss function in the generalized variable group sparse constraint source imaging model to ensure the robustness and accuracy of the brain power imaging result. In addition, the present invention also uses a deep learning model for solution, combining the constraint model with the deep learning model, which can effectively improve the solution efficiency of the ill-posed inverse problem.

[0010] In an alternative embodiment, the deep learning network model includes:

[0011] A convolutional layer for extracting local temporal features;

[0012] A bidirectional long short-term memory network, connected to the output end of the convolutional layer, for establishing long-term dependencies;

[0013] A spatial graph convolutional layer, connected to the output end of the bidirectional long short-term memory network, for encoding the electrode topology;

[0014] A dense mapping layer network module for mapping features to the meta-space dimension and reconstructing the brain power distribution.

[0015] In this embodiment, it has the ability to extract spatio-temporal features and fuse biophysical priors, which can effectively assist in solving the problem of low solution efficiency in the ill-posed inverse problem.

[0016] In an alternative embodiment, the generalized variable group sparse constraint source imaging model includes: a generalized variational regularization term model;

[0017] The generalized variational regularization term model is:

[0018]

[0019] where p≈Dx is the auxiliary variable corresponding to the first-order difference vector of the dipole strength, α is the regularization parameter for balancing the first-order and second-order dipole strength difference terms, E = D T , Ep is the second-order difference term describing the smoothness characteristic of the dipole strength, D is the linear transformation matrix for calculating the strength change between adjacent-order dipoles, and D is defined as:

[0020]

[0021] where,

[0022] where N e is the total number of edges formed by every two adjacent dipoles, and u is the unknown parameter for adjusting the conduction relationship between the lesion and the surrounding damaged and normal brain tissues.

[0023] In an alternative embodiment, the generalized variable group sparse constraint source imaging model further includes: a group sparse regularization term model;

[0024] The group sparse regularization term model is:

[0025]

[0026] where i is the neighborhood order, φ iis the serial number of the dipole included in the i-th order neighborhood.

[0027] In an alternative embodiment, the source imaging loss function is:

[0028]

[0029] where y is the multi-channel EEG signal, K is the prior of the conduction matrix, x is the intracranial nerve source signal estimated by the generalized variable group sparse constraint source imaging model, λ, α, β are regularization parameter, w d , w e , w c is the regularization weight.

[0030] In an alternative embodiment, preprocessing the multi-channel EEG signal includes:

[0031] Performing baseline correction on the multi-channel EEG signal, where the baseline correction includes removing the average value corresponding to the potential signal on each lead signal in the multi-channel EEG signal;

[0032] Dividing the multi-channel EEG signal by time into time periods, and removing the average value corresponding to each time point on each time period;

[0033] Normalizing the multi-channel EEG signal after removing the average value.

[0034] In a second aspect, the present invention provides a brain source imaging device, which includes:

[0035] An acquisition module for acquiring multi-channel EEG signals;

[0036] A processing module for preprocessing the multi-channel EEG signal;

[0037] A generation module for inputting the preprocessed multi-channel EEG signal into a pre-constructed generalized variable group sparse constraint source imaging model and outputting a brain source imaging result; in the generalized variable group sparse constraint source imaging model, the source imaging loss function is solved by a preset deep learning network model.

[0038] In a third aspect, the present invention provides a computer device, including: a memory and a processor, which are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the brain source imaging method according to the first aspect or any corresponding embodiment thereof.

[0039] In a fourth aspect, the present invention provides a computer-readable storage medium, which stores computer instructions for causing a computer to execute the brain source imaging method according to the first aspect or any corresponding embodiment thereof.

[0040] In a fifth aspect, the present invention provides a computer program product, including computer instructions for causing a computer to execute the brain power imaging method according to the first aspect or any corresponding embodiment thereof described above.

[0041] It should be noted that since the brain power imaging device, computer device, computer-readable storage medium, and computer program product provided by the present invention correspond to the above-mentioned brain power imaging method. Therefore, for the beneficial effects of the brain power imaging device, computer device, computer-readable storage medium, and computer program product, please refer to the description of the corresponding beneficial effects of the brain power imaging method above, and details will not be repeated here. Description of the Drawings

[0042] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 is a flowchart of the brain power imaging method according to an embodiment of the present invention;

[0044] Figure 2 is a network structure diagram of the brain power imaging according to an embodiment of the present invention;

[0045] Figure 3 is a structural block diagram of the brain power imaging device according to an embodiment of the present invention;

[0046] Figure 4 is a hardware structure diagram of the computer device according to an embodiment of the present invention. Detailed Embodiments

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0048] According to an embodiment of the present invention, an embodiment of a brain power imaging method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0049] In this embodiment, a brain power imaging method is provided, which can be executed by devices such as a server, a terminal, a mobile terminal, etc. Figure 1 It is a flowchart of the brain power imaging method according to an embodiment of the present invention, as Figure 1 shown, and the process includes the following steps:

[0050] Step S101, acquire multi-channel electroencephalogram (EEG) signals. The multi-channel EEG signals can be acquired by an EEG device. The EEG device is generally equipped with multiple electrodes, such as 32, 86, 128, etc., and can simultaneously acquire the electrical activities of multiple brain regions. Each electrode position will record the corresponding electrical activity, generating a set of multi-dimensional EEG signals.

[0051] Step S102, preprocess the multi-channel EEG signals. In this embodiment, data preprocessing can include baseline correction, power frequency interference elimination, normalization processing, etc.

[0052] Step S103, input the preprocessed multi-channel EEG signals into a pre-constructed generalized variable grouped sparse constraint source imaging model, and output the brain power imaging result; in the generalized variable grouped sparse constraint source imaging model, a preset deep learning network model is used to solve the source imaging loss function.

[0053] In this embodiment, the generalized variable grouped sparse constraint source imaging model includes a generalized variational regular term model and a group sparse regular term model. The generalized variational regular term model can describe the smoothness of the change in dipole strength within the neural electrical activity region and the large strength change at the connection between it and the region without electrical activity. The group sparse regular term model can describe the regionality of cortical electrical activity, that is, there must be other dipoles with non-zero strength in the region around a dipole with non-zero strength. Then, using the powerful modeling and optimization capabilities of deep learning, the network parameters are optimized through end-to-end learning, so as to more effectively solve complex optimization problems to achieve the solution of the source imaging loss function.

[0054] In this embodiment, according to the cortical discharge characteristics, a regular term constructed from two kinds of prior knowledge is introduced into the generalized variable grouped sparse constraint source imaging model as the loss function to ensure the robustness and accuracy of the brain power imaging result. In addition, in this embodiment, a deep learning model is also used for solution, and by combining the constraint model with the deep learning model, the solution efficiency of the ill-posed inverse problem can be effectively improved.

[0055] In some alternative embodiments, with reference to Figure 2 as shown, the deep learning network model includes:

[0056] a convolutional layer for extracting local temporal features;

[0057] a bidirectional long short-term memory network (Long Short-Term Memory, abbreviated as LSTM), connected to the output end of the convolutional layer, for establishing long-term dependencies;

[0058] a spatial graph convolutional layer, connected to the output end of the bidirectional long short-term memory network, for encoding the electrode topology;

[0059] a dense mapping layer network module for mapping features to the meta-space dimension and reconstructing the brain power distribution.

[0060] The deep learning network model in this embodiment adopts a spatio-temporal encoder-decoder architecture. Inputting multi-channel electroencephalogram signals (shape [Batch, Channels, Time]), it first extracts local temporal features through a 1D temporal convolutional layer, then models long-term dependencies through a bidirectional LSTM, and subsequently encodes the electrode topology through a spatial graph convolutional layer (GCN); in the fusion stage, it linearly projects the features to the source space dimension through a dense mapping layer network (Dense Layer) to reconstruct the brain power distribution, realizing an end-to-end mapping from electroencephalogram signals to cortical activities, with both spatio-temporal feature extraction and biophysical prior fusion capabilities, which can effectively assist in solving the problem of low solution efficiency in the ill-posed inverse problem.

[0061] In some alternative embodiments, the generalized variable group sparse constraint source imaging model includes: a generalized variational regularization term model;

[0062] The generalized variational regularization term model is:

[0063]

[0064] where p≈Dx is the auxiliary variable corresponding to the first-order difference vector of the dipole strength, α is the regularization parameter for balancing the first-order and second-order dipole strength difference terms, E = D T , Ep is the second-order difference term describing the smoothness characteristic of the dipole strength, D is the linear transformation matrix for calculating the strength change between adjacent-order dipoles, and D is defined as:

[0065]

[0066] where,

[0067] where, N eis the total number of edges formed by every two adjacent dipoles, and u is an unknown parameter for adjusting the conduction relationship between the lesion and the surrounding damaged and normal brain tissues.

[0068] In some optional embodiments, the generalized variable group sparse constraint source imaging model further includes: a group sparse regularization term model;

[0069] The group sparse regularization term model is:

[0070]

[0071] where i is the neighborhood order, and φ i is the dipole serial number included in the i-th order neighborhood.

[0072] In some optional embodiments, the source imaging loss function is:

[0073]

[0074] where y is the multi-channel electroencephalogram signal, K is the prior of the conduction matrix, x is the intracranial nerve source signal estimated by the generalized variable group sparse constraint source imaging model, λ, α, β are regularization term parameters, and w d , w e , w c are regularization term weights.

[0075] In some optional embodiments, preprocessing the multi-channel electroencephalogram signal includes:

[0076] Performing baseline correction on the multi-channel electroencephalogram signal, and the baseline correction includes removing the average value corresponding to the potential signal on each lead signal in the multi-channel electroencephalogram signal;

[0077] Dividing the multi-channel electroencephalogram signal by time into time periods, and removing the average value corresponding to each time point on each time period;

[0078] Normalizing the multi-channel electroencephalogram signal after removing the average value.

[0079] In this embodiment, first, baseline correction is performed on the signal of each lead, that is, subtracting its mean value to reduce baseline drift; secondly, subtracting the mean value of this time step at each time step to eliminate power frequency interference; finally, normalizing the entire EEG signal to facilitate model processing and comparison. These steps ensure that the EEG signal input into the model reduces the interference of environmental and equipment factors as much as possible, and can effectively improve the accuracy and reliability of model analysis.

[0080] In this embodiment, a brain power imaging device is further provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be elaborated again. As used hereinafter, the term "module" can be a combination of software and / or hardware that realizes a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0081] This embodiment provides a brain power imaging device, as Figure 3 shown, the device includes:

[0082] An acquisition module 201, configured to acquire multi-channel electroencephalogram signals;

[0083] A processing module 202, configured to preprocess the multi-channel electroencephalogram signals;

[0084] A generation module 203, configured to input the preprocessed multi-channel electroencephalogram signals into a pre-constructed generalized variable group sparse constraint source imaging model, and output a brain power imaging result; in the generalized variable group sparse constraint source imaging model, a preset deep learning network model is used to solve the source imaging loss function. Among them, the deep learning network model includes: a convolutional layer, configured to extract local time features; a bidirectional long short-term memory network, connected to the output end of the convolutional layer, configured to establish long-term dependencies; a spatial graph convolutional layer, connected to the output end of the bidirectional long short-term memory network, configured to encode the electrode topological structure; a dense mapping layer network module, configured to map the features to the meta-space dimension and reconstruct the brain power distribution.

[0085] In some optional implementation manners, the processing module 202 includes:

[0086] A processing unit, configured to perform baseline correction on the multi-channel electroencephalogram signals. The baseline correction includes removing the average value corresponding to the potential signal on each lead signal in the multi-channel electroencephalogram signals; dividing the multi-channel electroencephalogram signals by time into time periods, and removing the average value corresponding to each time point in each time period; and normalizing the multi-channel electroencephalogram signals after removing the average value.

[0087] The brain power imaging device in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0088] The further function descriptions of the above-mentioned various modules and units are the same as those in the corresponding above-mentioned embodiments, and will not be elaborated here.

[0089] This embodiment of the present invention further provides a computer device having the above-mentioned Figure 3 shown brain power imaging device.

[0090] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of a computer device provided by an alternative embodiment of the present invention. As Figure 4 shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some alternative embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a set of blade servers, or a multi-processor system). Figure 4 In

[0091] FIG. 9, one processor 10 is taken as an example.

[0092] The processor 10 can be a central processing unit, a network processor, or a combination thereof. Among them, the processor 10 can further include a hardware chip. The above hardware chip can be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The above programmable logic device can be a complex programmable logic device, a field programmable gate array, a general array logic, or any combination thereof.

[0093] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiments.

[0094] The memory 20 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the computer device. In addition, the memory 20 can include a high-speed random access memory and can also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 can optionally include a memory remotely set relative to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0094] The memory 20 can include volatile memory, such as random access memory; the memory can also include non-volatile memory, such as flash memory, a hard disk, or a solid-state drive; the memory 20 can also include a combination of the above types of memory.

[0095] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or communication networks.

[0096] Embodiments of the present invention also provide a computer-readable storage medium. The method according to the embodiments of the present invention can be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented as computer code that is originally stored in a remote storage medium or a non-transitory machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, the processor, or the hardware, the methods shown in the above embodiments are implemented.

[0097] A part of the present invention can be applied as a computer program product, such as computer program instructions. When executed by a computer, through the operation of the computer, the methods and / or technical solutions according to the present invention can be invoked or provided. Those skilled in the art should be able to understand that the forms of existence of computer program instructions in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Herein, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0098] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for brain power imaging, characterized in that: The method comprises: Acquire multi-channel EEG signals; Preprocessing the multi-channel EEG signal; The preprocessed multi-channel EEG signal is input into a pre-constructed generalized variable group sparse constrained source imaging model to output the EEG source imaging result; the source imaging loss function is solved in the generalized variable group sparse constrained source imaging model through a preset deep learning network model.

2. The method according to claim 1, characterized in that Deep learning network models include: Convolutional layer, used to extract local temporal features; A bidirectional long short-term memory network is connected to the output end of the convolutional layer to establish a long-term dependency relationship; A spatial graph convolution layer connected to the output end of the bidirectional long short-term memory network for encoding the electrode topology; The dense mapping layer network module is used to map features to metaspace dimensions and reconstruct brain power distribution.

3. The method according to claim 1, characterized in that: The generalized variational group sparse constrained source imaging model includes: a generalized variational regularization term model; The generalized variational regularization model is: Where p≈Dx is the auxiliary variable corresponding to the first-order difference vector of the dipole strength, α is the regularization parameter for balancing the first-order and second-order dipole strength difference terms, and E=D T , Ep is the second-order difference term describing the smooth characteristics of the dipole intensity, and D is the linear transformation matrix used to calculate the intensity change between adjacent dipoles. D is defined as: in, Among them, N e is the total number of edges formed by every two adjacent dipoles, and u is an unknown parameter that adjusts the conduction relationship between the lesion and the surrounding damaged brain tissue and normal brain tissue.

4. The method according to claim 3, characterized in that: The generalized variable group sparse constrained source imaging model also includes: a group sparse regularization term model; The group sparse regularization term model is: Among them, i is the neighborhood order, φ i is the dipole number contained in the i-th order neighborhood.

5. The method according to claim 4, characterized in that The source imaging loss function is: Where y is the multi-channel EEG signal, K is the conduction matrix prior, x is the intracranial neural source signal estimated by the generalized variable group sparse constrained source imaging model, λ, α, β are regularization term parameters, and w d , w e , w c is the regularization term weight.

6. The method according to claim 1, characterized in that The preprocessing of the multi-channel EEG signal comprises: Performing baseline correction on the multi-channel EEG signal, wherein the baseline correction includes removing the average value corresponding to the potential signal on each lead signal in the multi-channel EEG signal; Divide the multi-channel EEG signal into time periods according to time, and remove the average value corresponding to each time point in each time period; The multi-channel EEG signal after the average value is removed is normalized.

7. A brain power imaging device, characterized in that: The device comprises: An acquisition module, used for acquiring multi-channel EEG signals; A processing module, used for preprocessing the multi-channel EEG signal; A generation module is used to input the preprocessed multi-channel EEG signal into a pre-constructed generalized variable group sparse constrained source imaging model, in which the source imaging loss function is solved by a preset deep learning network model.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the brain power source imaging method according to any one of claims 1-6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the brain electrical source imaging method according to any one of claims 1-6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the brain electrical source imaging method according to any one of claims 1 to 6.

Images