Neuromorphic brain-computer emotion sensing method and device

Abstract

The invention provides a neuromorphic brain-computer emotion perception method and also provides corresponding equipment, and the method comprises the following steps: sampling electric signal data of a preset part of a brain through a graphene-silicon heterostructure sensor according to a preset frequency, and establishing a delayed sampling reserve pool; performing preliminary operation on the electric signal data of the delay sampling reserve pool and the position mask to obtain electric signal data with superposed spatial information; performing preliminary operation on the electric signal data superposed with the spatial information to obtain a weighted output connection weight; and making the computer equipment perform emotion judgment according to the weighted output connection weight, and outputting an emotion judgment result. According to the method, the network is built by using the position mask and the secondary reserve pool architecture, the problem of space-time fusion of electroencephalogram array signals is solved, the hardware network structure of the data acquisition side is greatly simplified, and simple and effective data calculation is performed on the equipment side after data acquisition, so that the operation efficiency is greatly improved.

Description

Technical Field

[0001] The present application relates to the field of brain-computer perception technology, and specifically to a neuromorphic brain-computer emotion perception method and corresponding equipment. Background Art

[0002] Highly dynamic autonomous decision-making is the core of the bionic intelligent perception system. How to construct efficient information acquisition and processing capabilities, break through the bottleneck of modern computers using the von Neumann architecture, and change the discrete architecture of signal sensing and computing are the research hotspots of neuromorphic computing and brain-computer interfaces.

[0003] The brains of higher organisms have strong asynchronous and parallel characteristics. Compared with artificially constructed neural networks, biological brains show super high energy utilization advantages. In recent years, with the research on distillation and quantization of artificial neural networks, it has also been verified that there is a large redundancy and room for improvement in artificial neural networks. In the field of traditional brain-computer interface and artificial intelligence, the classic research is to use artificial neural networks to analyze and understand the brain-computer interface signals that have been sampled and preprocessed. On the one hand, this scheme is limited by the destruction of the complexity of the signal space by structures such as the skull, and on the other hand, it is limited to the data labels of supervised learning, and it is difficult to give explanatory analysis and response to unknown physiological signal microstructures. Therefore, how to imitate the higher biological nervous system and construct a neuromorphic network with ultra-low energy consumption, support fast training with few samples, and highly imitate the structural characteristics of the biological nervous system has become an important direction of current neuromorphic brain-computer interface research. Summary of the invention

[0004] The purpose of this application is to solve the problem of how to imitate the higher biological nervous system and construct a neuromorphic network with ultra-low energy consumption, support fast training with a small number of samples, and highly imitate the structural characteristics of the biological nervous system.

[0005] In order to solve the above technical problems, the present application provides a neuromorphic brain-computer emotion perception method, comprising:

[0006] A neuron array composed of array unit neurons, each of which includes a graphene-silicon heterostructure sensor;

[0007] The method comprises:

[0008] Each graphene-silicon heterostructure sensor samples the electrical signal data of a preset location in the brain at a preset frequency to establish a delayed sampling reserve pool;

[0009] Two sets of orthogonal sinusoidal codes are used to perform mask mapping in two-dimensional space. Through the bias circuit of each graphene-silicon heterostructure sensor, a position mask with spatial information is superimposed on each array unit neuron.

[0010] Inputting the electrical signal data and the position mask in the delayed sampling storage pool into the secondary storage pool for preliminary calculation to obtain the electrical signal data with superimposed spatial information;

[0011] Inputting the electrical signal data of the superimposed spatial information into a readout network unit for preliminary calculation to obtain a weighted output connection weight;

[0012] The weighted output connection weight is output to a computer device, so that the computer device performs emotion judgment according to the weighted output connection weight and outputs an emotion judgment result.

[0013] Furthermore, the step of inputting the electrical signal data and the position mask in the delayed sampling storage pool into the secondary storage pool for preliminary calculation to obtain the electrical signal data of the superimposed spatial information includes:

[0014] The secondary storage pool receives the transient response values ​​of several virtual neuron nodes in the delayed sampling storage pool;

[0015] The transient response values ​​of the plurality of virtual neuron nodes are superimposed with the spatial information provided by the position mask to form an N-dimensional output tensor.

[0016] Furthermore, the secondary storage tank is composed of a graphene-silicon heterostructure sensor, and the graphene-silicon heterostructure sensor in the secondary storage tank is interconnected with the readout network unit through a small number of leads; the step of inputting the electrical signal data of the superimposed spatial information into the readout network unit for preliminary calculation to obtain the weighted output connection weight includes:

[0017] The N-dimensional output tensor and the weight matrix (W T ) performs matrix multiplication to obtain the weighted output connection weights.

[0018] Furthermore, the method also includes an electrical signal data acquisition unit connected to the graphene-silicon heterostructure sensor, wherein the electrical signal data acquisition unit acquires electrical signal data of a preset part of the brain and transmits the electrical signal data to the graphene-silicon heterostructure sensor.

[0019] In order to solve the above technical problems, the present application also provides a neuromorphic brain-computer emotion perception device, including:

[0020] A neuron array, a secondary storage tank and a readout network unit connected in sequence, wherein the neuron array includes array unit neurons composed of silicon-based graphene field effect sensors;

[0021] The device is used to execute the above method.

[0022] Furthermore, the secondary storage tank is composed of a graphene-silicon heterostructure sensor, and the graphene-silicon heterostructure sensor in the secondary storage tank is interconnected with the readout network unit through a small number of leads.

[0023] Furthermore, the array unit neuron also includes an electrical signal data acquisition unit, the electrical signal data acquisition unit includes an electroencephalogram signal probe, and the electrical signal data acquisition unit is used to execute the above method.

[0024] In order to solve the above technical problems, the present application also provides a neuromorphic brain-computer emotion perception method, including:

[0025] The computer device receives and reads the weighted output connection weight output by the network unit;

[0026] Substituting the weighted output connection weight into the trained emotion prediction model to perform emotion judgment;

[0027] Output the emotion judgment result.

[0028] Furthermore, the step of substituting the weighted output connection weight into the trained emotion prediction model for emotion judgment includes:

[0029] Substitute the weighted output connection weights into the trained emotion prediction model to determine the activation state of the membrane potential, and obtain the activation state of the membrane potential of the delayed sampling reserve pool at the set time;

[0030] If the membrane potential at the set moment is in an activated state, the trained ridge regression algorithm model is used to estimate and judge the weighted output connection weights to obtain the emotion prediction result;

[0031] The emotion prediction result is outputted.

[0032] Furthermore, the step of using the trained ridge regression algorithm model to estimate and judge the weighted output connection weights includes:

[0033] Comparing the weighted output connection weight with a trained emotion dataset, wherein the emotion dataset includes a plurality of emotion data;

[0034] Obtaining the closest emotion data and obtaining the emotion data label;

[0035] The emotion prediction result is determined according to the label.

[0036] In order to solve the above technical problems, the present application also provides a neuromorphic brain-computer emotion perception device, which includes a computer device, and the computer device is used to execute the above method.

[0037] The neuromorphic brain-computer emotion perception method provided in the present application not only has the characteristics of high-speed response and greatly improves the actual resolution, which is very suitable for EEG signal processing, but also realizes the construction of a delayed sampling reserve pool of array unit neurons, thereby realizing data acquisition of virtual multi-neuron nodes under the conditions of array unit neurons; and uses position masks and secondary reserve pool architecture to build a network to solve the problem of spatiotemporal fusion of EEG array signals, which not only greatly simplifies the hardware network structure on the data acquisition side, but also performs simple and effective data calculations on the device side after acquiring the data, thereby greatly improving the computing efficiency.

[0038] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specifically cites the preferred embodiments and describes them in detail with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a data flow diagram of the neuromorphic brain-computer emotion perception method according to an embodiment of the present application.

[0040] Figure 2 Schematic diagram of the graphene-silicon heterojunction sensor in the neuromorphic brain-computer emotion perception method of an embodiment of the present application.

[0041] Figure 3 This is a schematic diagram of the framework of the neuromorphic brain-computer emotion perception device according to an embodiment of the present application.

[0042] Figure 4 A network topology diagram using a location mask and a secondary reserve pool in combination is provided for an embodiment of the present application.

[0043] Figure 5 A schematic diagram of the structure of the electrodes and the derived circuit using the position mask and the secondary reservoir in the embodiment of the present application.

[0044] Figure 6 This is another data flow diagram of the neuromorphic brain-computer emotion perception method according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to further explain the technical means and effects adopted by the present application to achieve the intended application purpose, the present application is described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0046] Through the description of the specific implementation methods, a deeper and more specific understanding can be obtained of the technical means and effects adopted by the present application to achieve the predetermined purpose. However, the attached drawings are only for reference and illustration purposes and are not used to limit the present application.

[0047] Embodiment 1:

[0048] See also Figure 1 , Figure 2 , Figure 3 , Figure 4 and Figure 5 , embodiments of the neuromorphic brain-computer emotion perception method of the present invention include:

[0049] Figure 3 The schematic diagram of the hardware framework of the neuromorphic brain-computer emotion perception method of the present invention; in this embodiment, the hardware part includes a neuron array 10, a secondary storage tank 20 data-connected to the neuron array 10, a readout network unit 30 data-connected to the secondary storage tank 20, and a computer device 40 data-connected to the readout network unit 30. Among them, the neuron array 10 is composed of N array unit neurons, and the array unit neurons include an electrical signal data acquisition unit 101 and a graphene-silicon heterostructure sensor 102, and a bias circuit 103 is provided on the graphene-silicon heterostructure sensor 102; in this embodiment, the neuron array is a primary storage tank, also called a multi-node storage tank. The secondary storage tank is composed of a graphene-silicon heterostructure sensor. The graphene-silicon heterostructure sensor in the secondary storage tank is interconnected with the readout network unit through a small number of leads.

[0050] The graphene-silicon heterostructure sensor 102 is data-connected to the electrical signal data acquisition unit 101. The electrical signal data acquisition unit 101 of the neuromorphic brain-computer emotion perception device is attached to a certain emotional function part of the head of the tester who needs to perform brain-computer emotion perception. The electrical signal data acquisition unit 101 includes an electroencephalogram signal probe, and the electrical signal data acquisition unit 101 acquires electrical signal data of a preset part of the brain through the electroencephalogram signal probe.

[0051] See also Figure 2 Graphene-silicon heterostructure sensor is a field effect coupling device, which is often used in the field of broadband photoelectric sensors. It uses a probe to connect the field effect structure as an electrode to realize the delayed sense internal reserve pool structure of a single neuron. The field effect coupling device uses a deeply depleted silicon well to realize charge accumulation, and the single-layer graphene structure can directly read the result without loss. The whole structure is a typical graphene field effect tube. The reason for choosing silicon as a deep well is to take advantage of its high resistivity. Holes and excited electrons are captured by the silicon well and the graphene readout gate respectively. The hole capture process of the silicon well is linear, and the drain current of the p-doped graphene will be reduced, thereby reading the detection results. The high-speed response frequency of the device structure is above 1MHz, with high time resolution, which is very suitable for EEG signal processing. All sensors in the array are read randomly, which can well support asynchronous operations.

[0052] Specifically, Figure 2 The graphene-silicon heterostructure sensor structure shown in can be used as a single-node leaky integrated-and-fire (LIF) neuron. The specific integration process is: when the electrical signal stimulation is input, the potential will gradually accumulate in the well; the leakage process is: when the electrical signal stimulation is withdrawn, the accumulated potential will be gradually released, and the accumulated potential can be directly read from the graphene structure as the neuron sampling at that moment. When the threshold is exceeded, the two ends can be short-circuited to quickly release the potential as a neuron activation signal. In the delayed sampling reservoir structure of the array unit neuron, the activation effect can be simplified, so that the array unit neuron works in an unsaturated state, and the delayed sampling reservoir of the array unit neuron is made by the nonlinearity of the integration and leakage process. The graphene-silicon heterostructure sensor is used as a single-node LIF neuron to apply electrical stimulation and read out at a fixed sampling rate, and the well potential is freely leaked between two sampling intervals to simulate the leakage process of the biological neuron membrane potential.

[0053] See also Figure 1 , Figure 3 , Figure 4 and Figure 5 ,The data flow of the neuromorphic brain-computer emotion perception method of the present invention is as follows;

[0054] 101. Obtaining electrical signal data;

[0055] After each electrical signal data acquisition unit 101 in the array unit neuron acquires the electrical signal data of a preset part of the brain through an EEG signal probe, it transmits the data to the corresponding graphene-silicon heterostructure sensor.

[0056] 102. Storing electrical signal data and establishing a delayed sampling reserve pool;

[0057] When sampling the electrical signal data of a preset part of the brain, each graphene-silicon heterostructure sensor 102 performs sampling at a predetermined frequency, temporarily stores the sampled electrical signal data locally, and establishes a delayed sampling reserve pool.

[0058] 103. Each array unit neuron is superimposed with a position mask with spatial information;

[0059] See also Figure 4 , two orthogonal sets of sinusoidal codes are used to perform mask mapping in two-dimensional space, and a position mask with spatial information is superimposed on each array unit neuron through the bias circuit 103 of each graphene-silicon heterostructure sensor; please refer to the bias circuit Figure 2 ;

[0060] 104. Input the electrical signal data and the position mask in the delayed sampling storage pool into the secondary storage pool for preliminary calculation to obtain the electrical signal data with superimposed spatial information;

[0061] See also Figure 4 After the electrical signal data in the delayed sampling reserve pool reaches a predetermined number, it is combined with the corresponding position mask and input into the secondary reserve pool 20 for preliminary calculation to obtain the electrical signal data of the superimposed spatial information, that is, to form an N-dimensional output tensor of the superimposed spatial information, which is output to the readout network unit 30.

[0062] 105. Inputting the electrical signal data of the superimposed spatial information into the readout network unit for preliminary calculation to obtain the weighted output connection weight;

[0063] The readout network unit 30 receives the electrical signal data of the superimposed spatial information, namely the N-dimensional output tensor of the superimposed spatial information, and the preset parameter matrix W of the readout network unit 30 T Perform multiplication and addition operations to obtain the weighted output connection weights.

[0064] 106. Output the weighted output connection weight to the computer device 40;

[0065] After obtaining the weighted output connection weight, the weighted output connection weight is output to the computer device 40, wherein the computer device 40 is pre-set with a trained neuromorphic brain-computer emotion perception model, and the weighted output connection weight is substituted into the pre-trained emotion perception model for calculation and analysis, and the emotion judgment result is predicted and output.

[0066] In this embodiment, this method, by setting the use of graphene-silicon heterostructure sensors in the brain-computer emotion perception device, not only has the characteristics of high-speed response, greatly improving the actual resolution and being very suitable for EEG signal processing; it also realizes the construction of a delayed sampling reserve pool of array unit neurons, thereby realizing data acquisition of virtual multi-neuron nodes under the conditions of array unit neurons; uses position masks and secondary reserve pool architecture to build a network to solve the problem of spatiotemporal fusion of EEG array signals, which not only greatly simplifies the hardware network structure on the data acquisition side, but also performs simple and effective data calculations on the device side after acquiring the data, thereby greatly improving the computing efficiency.

[0067] Embodiment 2:

[0068] The present invention discloses a neuromorphic brain-computer emotion perception method, specifically:

[0069] 201. Receive weighted output connection weights;

[0070] The computer device receives and reads the weighted output connection weight output by the network unit;

[0071] 202. Substituting the weighted output connection weight into the trained emotion prediction model to perform emotion judgment;

[0072] The weighted output connection weight is substituted into the trained emotion prediction model to judge the activation state of the membrane potential, and the activation state of the membrane potential of the delayed sampling reserve pool at the set time is obtained; if the membrane potential at the set time is in the activated state, the trained ridge regression algorithm model is used to estimate and judge the weighted output connection weight, specifically including: comparing the weighted output connection weight with the trained emotion data set, wherein the emotion data set includes multiple emotion data; obtaining the emotion data closest to the weighted output connection weight, and obtaining the emotion data label; and then judging the emotion prediction result based on the label.

[0073] The specific training process of the emotion prediction model is described in detail in Example 3 and will not be repeated here.

[0074] 203. Output the emotion judgment result.

[0075] In this embodiment, by establishing and training the aforementioned emotion perception model on the computer side, the weighted output connection weights obtained by weighted summation operation of multiple electrical signal data acquired by the electrical signal data acquisition unit in the graphene-silicon heterostructure sensor are compared with the trained emotion data set, and the label closest to the emotion data is obtained to obtain the emotion prediction result; the response speed is greatly improved.

[0076] Embodiment three:

[0077] The present invention discloses an application example of a neuromorphic brain-computer emotion perception method, specifically:

[0078] The electrical signal data acquisition unit of the neuromorphic brain-computer emotion perception device is attached to a certain emotional function part of the head of a tester who needs to perform brain-computer emotion perception. The electrical signal data acquisition unit includes an electroencephalogram (EEG) signal probe, and the electrical signal data acquisition unit acquires electrical signal data of a preset part of the brain through the EEG signal probe.

[0079] The neuromorphic brain-computer emotion perception device also includes a graphene-silicon heterostructure sensor, which is data-connected to the electrical signal data acquisition unit. The electrical signal data acquisition unit acquires the electrical signal data of the preset part of the brain through the EEG signal probe and transmits it to the graphene-silicon heterostructure sensor; when sampling the electrical signal data of the preset part of the brain, the graphene-silicon heterostructure sensor samples at a predetermined frequency, temporarily stores the sampled electrical signal data locally, and establishes a delayed sampling reserve pool;

[0080] Two sets of orthogonal sinusoidal codes are used to perform mask mapping in two-dimensional space. Through the bias circuit of each graphene-silicon heterostructure sensor, a position mask with spatial information is superimposed on each array unit neuron.

[0081] When the electrical signal data in the delayed sampling storage pool reaches a predetermined amount, it is combined with the corresponding position mask and input into the secondary storage pool for storage operation to form an N-dimensional output tensor of superimposed spatial information. The N-dimensional output tensor of superimposed spatial information is input into the readout network unit, and the readout network unit performs a preliminary operation on the input N-dimensional output tensor to obtain the weighted output connection weight. The preliminary operation here includes: combining the N-dimensional output tensor with W T Multiplication and addition operations are performed to obtain weighted output connection weights, and the weighted output connection weights are output to a computer device, wherein the computer device is pre-set with a trained neuromorphic brain-computer emotion perception model, and the weighted output connection weights are substituted into the pre-trained emotion perception model for calculation and analysis, and the emotion judgment results are predicted and output.

[0082] The process of establishing and training the aforementioned emotion perception model is as follows:

[0083] The establishment and training of the emotion perception model is divided into two parts, the first part is on the graphene-silicon heterostructure sensor side, and the second part is on the computer equipment side.

[0084] By fusing the classic dataset and the actual sampling dataset, an emotion dataset is established and dataset labels are set. The labels are used to classify the emotion data. Specifically: simulation is performed on a graphene-silicon heterostructure sensor, direct feeding is performed using the classic dataset, the readout network unit is trained to complete the emotion recognition task, an emotion dataset is established and dataset labels are set. The labels are used to classify the emotion data; training parameters are downloaded, and the emotion recognition task is verified using real data, an emotion dataset is established and dataset labels are set; and the parameters of the readout network unit are set according to the emotion dataset label, and the performance of the emotion recognition brain-computer interface is debugged as a whole.

[0085] The mathematical model of the reserve pool is established as follows:

[0086] x(t)=sig(W x x(t-1)+W in u(t))……(1)

[0087] W in is the connection weight input to the primary pool. In the multi-node pool, this value is a random matrix with an average distribution of 0-1, which maps the data of n sensors to m neurons. in The dimension of W is n*m. xis the node interconnection matrix in the primary reservoir, with a dimension of m*m, where m is the number of neurons in the multi-node reservoir. x(t-1) is the membrane potential state of the primary reservoir at time t-1, and u(t) is the input value of the sensor at time t. sig is a sign function, used as an activation function. When the independent variable is greater than or equal to zero, the dependent variable is 1, and when the independent variable is less than zero, the dependent variable is 0. It is used to simulate the activation state of neurons.

[0088] Substitute the membrane potential state x(t) of the delayed sampling reserve pool at time t in formula (1) into formula (2) to calculate the emotional problem description data, that is, the weight W connected to the weighted output out (t) performing matrix multiplication operations;

[0089] Specific:

[0090] y(t)=W out x(t)+b……(2)

[0091] The readout network unit training of the delayed sampling reserve pool adopts the ridge regression algorithm, and finally adjusts the output connection weight. Specifically, the process of training the delayed sampling reserve pool calculation is to calculate the output connection weight W of the weighted sum of the transient responses of N virtual nodes at time t. out (t) is as close as possible to the target matrix y(t); and the emotional problem description data is obtained. In this embodiment, b=0.

[0092] Among them, read out the final output connection weight W of the network unit out (t) can be expressed by the following formula:

[0093] W out (t) = argmin || y(t) - W out x(t)|| 2 +λ||W out || 2 ……(3)

[0094] Where x(t) is the sum of the transient responses of all virtual nodes in the delayed sampling reserve pool, W out (t) is the weighted output connection weight W obtained by weighted summation of the transient responses of N virtual nodes at time t. out (t), since we only focus on the state at time t, we will out (t) is denoted as W out , λ is the ridge parameter in the ridge regression algorithm.

[0095] Finally, the weighted connection weight W calculated by the read network unit is out The data is sent to the computer for offline training. The mathematical formula of the training model is:

[0096]

[0097] Where X is x(t) in formula (3), Y is y(t) in formula (3), and I is the unit matrix. During training, the weighted connection weight W is adjusted by continuously adjusting the operation parameters. out Optimize so that the optimized weighted connection weight W out Getting closer and closer to the target weight connection weight, that is, the weighted connection weight W out The final estimate of .

[0098] After the emotion perception model is trained, the electrical signal data of the array unit neurons are sampled with a fixed step length. Each sampling is used as a virtual node, and multiple sampling nodes form a delayed sampling reserve pool; two sets of orthogonal sinusoidal codes are used to perform mask mapping in two-dimensional space, and a position mask with spatial information is superimposed on each array unit neuron through the bias circuit 103 of each graphene-silicon heterostructure sensor; the electrical signal data and the position mask of the delayed sampling reserve pool are then input into the secondary reserve pool for preliminary calculation to obtain the electrical signal data with superimposed spatial information; the electrical signal data with superimposed spatial information is input into the readout network unit for preliminary calculation, the weighted connection weight is output, and the trained emotion perception model is used for judgment, and finally the emotion judgment result is output.

[0099] In one embodiment, in a classic 64-channel EEG sensor array, the electrodes are divided into an 8*8 array and a 3-bit horizontal and vertical position mask is applied. After the horizontal and vertical masks are spliced, they are directly linearly superimposed on the output of the internal storage pool of the graphene field effect sensor array unit node delay sampling.

[0100] The secondary reserve pool still uses the same graphene field-effect device, the only difference is that it is not connected to the EEG probe.

[0101] Specifically: For example, in a classic 64-channel array, an 8*8 method is used to superimpose position masks. For example, if an array unit is in the i-th row and j-th column, a normalized voltage of A(0.5*cos(i / 8*π / 2)+0.5*cos(j / 8*π / 2)) is added to the neuron of the array. The typical value of the factor A should be half of the neuron membrane potential threshold. This ensures that a basic potential is superimposed on the neuron that will not be easily activated or saturated. In the operation of the array unit neuron network, the membrane potential of the array unit neurons is collected N times as N virtual neuron nodes, and then the N sampling points are recorded as s1 s2 s3... sN, forming an output tensor as the delayed sampling reserve pool state response output. The N-dimensional output tensor is combined with W TPerform matrix multiplication to obtain the final output of m-dimensional data (e.g., classification as epilepsy or normal, m=2). Reading the preset parameter matrix of the network unit is a matrix multiplication and an adder. In actual engineering use, a programmable logic device (Field-Programmable Gate Array, FPGA) is used for simulation. That is, the formula y(t)=W out The m-dimensional data in (t)+b is output to the pre-trained readout network unit for data judgment to obtain the emotional question description data, that is, the weighted output connection weight, and then the weighted output connection weight is brought into the emotional prediction model for analysis to finally obtain the emotional judgment result.

[0102] For example, in the monitoring scenario of neurodegenerative diseases, the emotion dataset is the classification label of EEG and neurodegenerative stages. Assuming that 10s is used as the time window, neurodegenerative diseases can be classified after training. In actual deployment, a 10s time window and a time step of sampling once every 1 minute are used to collect the electrical signal dataset in units of 10 windows. The classification labels are determined according to the model inference results, and the final prediction value of the model is determined by voting. Thus, the monitoring of neurodegenerative diseases is achieved. In actual use, parameter scanning or fine-tuning is performed on hyperparameters such as window length, sampling step, and the final number of sampling windows. The model inference accuracy is compared according to the model inference results and other medical indicators, and the optimal inference hyperparameters are selected to improve the inference strategy. Among them, 10s, 1min, and 10 windows are values ​​taken based on experience, which need to be adjusted in actual applications to improve the performance of the emotion recognition brain-computer interface.

[0103] In the embodiment of the present invention, by setting the use of graphene-silicon heterostructure sensors in the brain-computer emotion perception device of the array unit neurons, it not only has the characteristics of high-speed response, greatly improves the actual resolution and is very suitable for EEG signal processing; it also realizes the construction of a delayed sampling reserve pool of the array unit neurons, thereby realizing the data acquisition of virtual multi-neuron nodes under the conditions of the array unit neurons; it not only greatly simplifies the hardware network structure on the data acquisition side, but also performs simple and effective data calculations on the device side after acquiring the data, thereby greatly improving the computing efficiency.

[0104] Embodiment 4:

[0105] See also Figure 3The neuromorphic brain-computer emotion perception device in this embodiment includes a neuron array 10, a secondary storage tank 20 and a readout network unit 30 which are data-connected in sequence. The neuron array includes N array unit neurons composed of silicon-based graphene field effect sensors; the array unit neurons include an electrical signal data acquisition unit 101 and a graphene-silicon heterostructure sensor 102; the secondary storage tank is composed of a graphene-silicon heterostructure sensor, and the graphene-silicon heterostructure sensor in the secondary storage tank is interconnected with the readout network unit through a small number of leads.

[0106] See also Figure 5 , the blue unit represents the EEG probe applied to the graphene-silicon heterostructure sensor, and the position mask is superimposed through the pre-set peripheral circuit. The gray unit is a distributed secondary reservoir. Due to the flexibility of the reservoir connection process, the signal connection here is only used as an example. The neuronal signal connection degree of the secondary reservoir can be reduced or increased to adjust the hyperparameters of the reservoir. The secondary reservoir here still uses the same graphene field effect device, the only difference is that there is no EEG probe connected. This device consistency will be beneficial to integration and manufacturing.

[0107] See also Figure 5 , the array edge neurons only lead out a small number of leads to the readout network unit for matrix calculation. Thanks to the design of the position mask, the spatial information of the array has been integrated in the secondary reserve pool, and there is no need to lead out all units in the vertical plane, nor to use the bus readout of the readout circuit, which greatly simplifies the hardware structure.

[0108] The graphene-silicon heterostructure sensor 102 is data-connected to the electrical signal data acquisition unit 101 . The graphene-silicon heterostructure sensor 102 includes a silicon-based graphene field effect sensor and an electrode array.

[0109] The electrical signal data acquisition unit 101 of the neuromorphic brain-computer emotion perception device is attached to a certain emotional function part of the head of a tester who needs to perform brain-computer emotion perception. The electrical signal data acquisition unit 101 includes an electroencephalogram (EEG) signal probe, and the electrical signal data acquisition unit 101 acquires electrical signal data of a preset part of the brain through the EEG signal probe.

[0110] Neuromorphic brain-computer emotion sensing devices are used to perform Figure 1 The instructions described in are not repeated here.

[0111] In the embodiment of the present invention, by setting the use of graphene-silicon heterostructure sensors in the brain-computer emotion perception device, it not only has the characteristics of high-speed response, but also greatly improves the actual resolution and is very suitable for EEG signal processing; it also realizes the construction of a delayed sampling reserve pool of array unit neurons, thereby realizing data acquisition of virtual multi-neuron nodes under the conditions of array unit neurons; using position masks and secondary reserve pool architecture to build a network to solve the problem of spatiotemporal fusion of EEG array signals, which not only greatly simplifies the hardware network structure on the data acquisition side, but also performs simple and effective data calculations on the device side after acquiring the data, thereby greatly improving the computing efficiency.

[0112] Considering the signal quality problem, the classic brain-computer interface needs to digitally filter the signal sampled by the electrode in the upper computer to remove interference such as AC mains, muscle electricity, and electro-oculographic artifacts. In the present invention, the volatile conductivity effect of the graphene field effect tube itself is equivalent to a low-pass filter, and the parameters such as the doping structure can be designed to adjust the response curve to directly shield the high-frequency mains interference. For problems such as artifacts, end-to-end operations can be performed directly in the reserve pool operation without the need to use classic wavelet filtering or empirical mode decomposition methods.

[0113] Embodiment five:

[0114] See also Figure 3 The neuromorphic brain-computer emotion perception device in this embodiment includes: a computer device 20, in which an emotion perception model as in Example 3 is established and trained.

[0115] The computer device 20 is used to execute Figure 6 The instructions described in are not repeated here.

[0116] In the embodiment of the present invention, by setting the use of graphene-silicon heterostructure sensors in the brain-computer emotion perception device of the array unit neurons, it not only has the characteristics of high-speed response, but also greatly improves the actual resolution and is very suitable for EEG signal processing; it also realizes the construction of a delayed sampling reserve pool of the array unit neurons, thereby realizing the data acquisition of virtual multi-neuron nodes under the conditions of array unit neurons; the network is built by using the position mask and the secondary reserve pool architecture to solve the problem of spatiotemporal fusion of EEG array signals, which not only greatly simplifies the hardware network structure on the data acquisition side, but also performs simple and effective data calculations on the device side after acquiring the data, thereby greatly improving the computing efficiency.

[0117] The above is only a preferred embodiment of the present application and does not constitute any form of limitation to the present application. Although the present application has been disclosed as a preferred embodiment as above, it is not intended to limit the present application. Any technician familiar with the profession can make some changes or modifications to equivalent embodiments of the technical contents disclosed above without departing from the scope of the technical solution of the present application. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. A neuromorphic brain-computer emotion perception method, characterized in that: The method includes a neuron array composed of array unit neurons, each array unit neuron comprising a graphene-silicon heterostructure sensor; The method comprises: Each graphene-silicon heterostructure sensor samples the electrical signal data of a preset location in the brain at a preset frequency to establish a delayed sampling reserve pool; Two sets of orthogonal sinusoidal codes are used to perform mask mapping in two-dimensional space. Through the bias circuit of each graphene-silicon heterostructure sensor, a position mask with spatial information is superimposed on each array unit neuron. Inputting the electrical signal data and the position mask in the delayed sampling storage pool into the secondary storage pool for preliminary calculation to obtain the electrical signal data with superimposed spatial information; Inputting the electrical signal data of the superimposed spatial information into a readout network unit for preliminary calculation to obtain a weighted output connection weight; The weighted output connection weight is output to a computer device, so that the computer device performs emotion judgment according to the weighted output connection weight and outputs an emotion judgment result.

2. The method according to claim 1, characterized in that The step of inputting the electrical signal data and the position mask in the delayed sampling storage pool into the secondary storage pool for preliminary calculation to obtain the electrical signal data with superimposed spatial information comprises: The secondary storage pool receives the transient response values ​​of several virtual neuron nodes in the delayed sampling storage pool; The transient response values ​​of the plurality of virtual neuron nodes are superimposed with the spatial information provided by the position mask to form an N-dimensional output tensor.

3. The method according to claim 2, characterized in that The step of inputting the electrical signal data of the superimposed spatial information into the readout network unit for preliminary calculation to obtain the weighted output connection weight includes: The N-dimensional output tensor and the weight matrix (W T ) performs matrix multiplication to obtain the weighted output connection weights.

4. The method according to claim 1, characterized in that The method also includes an electrical signal data acquisition unit connected to the graphene-silicon heterostructure sensor, wherein the electrical signal data acquisition unit acquires electrical signal data of a preset part of the brain and transmits the electrical signal data to the graphene-silicon heterostructure sensor.

5. A neuromorphic brain-computer emotion perception method, characterized in that: include: The computer device receives and reads the weighted output connection weight output by the network unit; Substituting the weighted output connection weight into the trained emotion prediction model to perform emotion judgment; Output the emotion judgment result.

6. The method according to claim 5, characterized in that The step of substituting the weighted output connection weight into the trained emotion prediction model for emotion judgment comprises: Substitute the weighted output connection weights into the trained emotion prediction model to determine the activation state of the membrane potential, and obtain the activation state of the membrane potential of the delayed sampling reserve pool at the set time; If the membrane potential at the set moment is in an activated state, the trained ridge regression algorithm model is used to estimate and judge the weighted output connection weights to obtain the emotion prediction result; The emotion prediction result is outputted.

7. The method according to claim 6, characterized in that The step of using the trained ridge regression algorithm model to estimate and judge the weighted output connection weights comprises: Comparing the weighted output connection weight with a trained emotion dataset, wherein the emotion dataset includes a plurality of emotion data; Obtaining the closest emotion data and obtaining the emotion data label; The emotion prediction result is determined according to the label.

8. A neuromorphic brain-computer emotion perception device, characterized in that: The device comprises a neuron array, a secondary storage tank and a readout network unit which are sequentially data-connected, wherein the neuron array comprises array unit neurons composed of silicon-based graphene field effect sensors; The device is used to execute the method according to any one of claims 1 to 4.

9. The device according to claim 8, characterized in that The secondary storage tank is composed of a graphene-silicon heterostructure sensor, and the graphene-silicon heterostructure sensor in the secondary storage tank is interconnected with the readout network unit through a small number of leads.

10. A neuromorphic brain-computer emotion perception device, characterized in that: The device comprises a computer device, and the computer device is used to execute the method according to any one of claims 5 to 7.

Images