Auditory attention detection method based on pulse inspiration

Through the auditory attention detection method based on pulsed neural network, the human brain information processing mechanism is simulated, and the problems of poor interpretability and low real-time in the existing technology are solved, and auditory attention detection with high accuracy and low latency are achieved. Sound detection in noisy environments.

CN119046753AActive Publication Date: 2024-11-29DALIAN UNIV OF TECH

Patent Information

Application Number
CN202411084249.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-29
Estimated Expiration
2044-08-08

AI Technical Summary

Technical Problem

The existing auditory attention detection methods based on EEG signals have problems such as poor interpretability, low real-time and low accuracy, especially in noisy environments, which are difficult to effectively detect the sounds of users' attention.

Method used

The auditory attention detection method based on pulse neural network is adopted, and the human brain information processing mechanism is simulated, and the electroencephalogram signals and audio information are encoded by synchronously collected. The characteristics are extracted in combination with the pulse layer and the channel attention module, and the feature fusion module is performed through the feature fusion module. Classify to achieve higher accuracy and generalization.

Benefits of technology

Achieving attention detection with low latency and high accuracy is achieved, with better generalization and stability, able to maintain consistency of detection results on different subjects, and significantly reduce model energy when combined with specialized neural hardware. Consumption.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119046753A_ABST
    Figure CN119046753A_ABST
Patent Text Reader

Abstract

The invention relates to the field of brain-computer interfaces in the interdiscipline of computers and neuroscience, and provides an auditory attention detection method based on pulse inspiration. A threshold value calculation module in the method provides a threshold value for a pulse layer, and an electroencephalogram branch and an audio branch are used for processing time-synchronized electroencephalogram signals and voice envelopes respectively. The electroencephalogram branch extracts time features by using a pulse layer and encodes the time features, wherein a channel attention module for electroencephalogram is arranged; according to the audio branch, two input audios are subjected to feature extraction by using a pulse layer and are coded. A feature fusion module is used for extracting fusion features of the electroencephalogram and the two audios; and finally, obtaining an auditory attention detection result by using a classifier. Experiments show that the method realizes high accuracy under low delay and short decision window, has better generalization, and has stable detection results on different subjects.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interface in computer science, and in particular to an auditory attention detection method based on electroencephalogram data and audio. Background Art

[0002] When there are multiple speakers speaking at the same time in a noisy environment, how can people separate the voice of the speaker they are interested in from the background noise and understand it is called the cocktail party problem. Since it was proposed in 1953, the cocktail party problem has been a hot area of ​​research. Although people with normal hearing can easily analyze complex speech scenes by focusing their attention and using spatial positioning, some people with hearing loss (such as sensorineural hearing loss) find it extremely challenging to communicate in an acoustic environment with multiple speakers even if they wear hearing aids or cochlear implants. According to the latest Global Burden of Disease study results, the burden caused by hearing loss has increased over time due to the aging population, so the global demand for hearing aids is also increasing. In the past decade, hearing aids have made substantial progress in suppressing background noise that interferes with speech. They can reduce simple environmental noise and amplify the sound source in front of the user. There are also automatic speaker separation and speech enhancement algorithms with good performance, but they cannot enhance the target speaker without knowing which speaker the listener is talking to. Therefore, it is very important to detect the sound that the user is paying attention to in a noisy environment.

[0003] Recent studies of the properties of speech representations in the human auditory cortex have found that attended speakers have stronger representations relative to unattended sound sources. These findings raise the prospect of a brain-controlled assistive listening device that continuously monitors the listener's brain waves and compares them to the sounds in the environment to determine the speaker the subject is most likely attending to. The device could then amplify the attended speaker's voice relative to others so that the speaker can be better heard in a crowd. This process is called Auditory Attention Decoding (AAD), and this area of ​​research has seen considerable growth in recent years.

[0004] Auditory attention detection method is a typical application of brain-computer interface technology, which is of great significance in the study of brain function and neuroscience. In addition, in the fields of emotional computing, speech recognition, natural language processing, etc., researchers have made important progress through further exploration of brain function and neuroscience. Studying auditory attention detection method can not only solve the cocktail party problem and develop brain-controlled hearing aids, but also provide a certain foundation for in-depth research in brain science.

[0005] At present, the research on AAD based on EEG signals at home and abroad can be mainly divided into two categories: linear method and nonlinear method.

[0006] Linear methods use various modeling methods to find the connection between EEG and speech, which can be roughly divided into forward modeling (modeling of speech to EEG) and backward modeling (modeling of EEG to speech). Stimulus reconstruction is a classic algorithm for backward modeling. This method applies a neural decoder to the EEG channel to reconstruct the participating speech envelope or spectrogram, and then compares it with the speech representation of all speakers. The speaker with the highest Pearson correlation coefficient is identified as the present speaker. The AAD method based on the time domain response function belongs to forward modeling. The EEG signal is estimated through speech, and then the estimated EEG signal is correlated with the real signal, and finally classified according to the correlation. There are also methods based on canonical correlation analysis, which perform forward modeling and backward modeling at the same time. The above-mentioned linear methods all have the problem of low correlation between the reconstructed data and the original data. They assume that there is a linear relationship between the EEG and speech signals, and cannot fully fit the nonlinear characteristics of the auditory system.

[0007] Nonlinear methods have been proposed in recent years, usually based on deep learning frameworks to achieve auditory attention detection. For example, based on convolutional neural networks, long short-term memory networks, attention networks, etc., these networks can learn the nonlinear relationship between EEG signals and audio, better simulate the process of human brain processing data, and solve the problems of poor real-time performance and low accuracy of linear models. However, the general deep learning methods used in the research on auditory attention detection using neural networks do not take into account the characteristics of EEG signals and neurocognitive mechanisms, resulting in the "black box" problem and poor model interpretability.

[0008] In contrast, the present invention introduces a biologically inspired spiking neural network that simulates the workings of biological nervous systems, is more interpretable, can process time series data well, and has lower energy consumption. Summary of the invention

[0009] In view of the shortcomings of the existing methods, the present invention proposes a pulse-inspired auditory attention detection method, which can effectively complete short-sequence auditory attention detection. The method is based on deep learning and uses synchronously collected EEG signals and audio information to perform short-sequence auditory attention detection. The EEG signals and audio information are encoded by simulating the information processing mechanism of the human auditory system in the perception process through a pulse neural network, and then input into the feature extraction network on the back side to complete the classification.

[0010] The technical solution of the present invention is:

[0011] A pulse-inspired auditory attention detection method, wherein the synchronously collected EEG signal and two audios are preprocessed, and the pulse neural network threshold is first calculated through the EEG signal, and then respectively input into the EEG branch and the audio branch; the EEG branch first extracts the time feature through the pulse layer and completes the encoding, and then the channel attention module is applied to capture the channel relationship of the EEG signal; if there are two synchronous voices, there are audio branches that process the two audios respectively, and the sequence time features are extracted and encoded through the pulse layer respectively; the outputs of the two audio branches are spliced ​​with the outputs of the EEG branch in the channel dimension, and then the splicing results are respectively input into the feature fusion module to obtain the fusion features between the EEG and the two audios, and the fully connected layer maps the fusion features to the output dimension with the same number of categories, and finally the classification result is obtained through the softmax function.

[0012] The specific steps are as follows:

[0013] Step 1: Preprocessing of EEG signals and audio

[0014] The EEG signals were first high-pass filtered with a cutoff frequency of 0.5 Hz to eliminate DC components and electrode drift; then the EEG signals were re-referenced to the EEG Cz channel and down-sampled to 128 Hz and band-pass filtered.

[0015] The audio is processed using an auditory filter with power-law compression to obtain the speech envelope, and finally downsampled to 128 Hz and bandpass filtered.

[0016] In order to make the data more suitable for subsequent networks, the EEG signals are standardized and normalized, and the speech envelope is normalized; in order to effectively verify the detection accuracy of the invention in a short time, a sliding window is used to divide the EEG signals and speech envelopes into continuous decision windows, and the synchronization of the EEG signals and the audio is always maintained.

[0017] Step 2: Calculate the threshold of the spiking neural network based on the EEG signal

[0018] The threshold calculation is to provide the subsequent pulse layer with a threshold that is more suitable for different subjects to improve the generalization and robustness of the network.

[0019] The threshold is calculated based on the given EEG data. The EEG signal is input into the threshold calculation module after a period of training. Then, the threshold is fixed after a period of training to train other network layers.

[0020] The threshold calculation module first calculates the mean and standard deviation of each channel of the input EEG signal E, then merges the mean and standard deviation together to obtain a new tensor u, which is then input into the convolution layer, and then batch normalized. The Sigmoid function is applied to map the value to the range of (0,1), and then the average value is calculated to obtain the spike neural network threshold g, which is expressed as follows:

[0021] u=Concat(Mean(E),Std(E)) (1)

[0022] g=Avg(σ(BN(Conv(u)))) (2)

[0023] Among them, Mean means calculating the mean, Std means calculating the standard deviation, Concat means connecting in the channel dimension, Conv means one-dimensional convolution layer, BN means batch normalization layer, σ means Sigmoid activation layer, Avg means averaging, and g means the pulse neural network threshold.

[0024] Step 3: Input the preprocessed EEG signal and audio into the EEG branch and audio branch respectively

[0025] The EEG branch is based on the Spike-CNN architecture and includes a spike layer and a channel attention module. In order to better utilize the temporal feature extraction capability of spike neurons, the input EEG signal is divided into K consecutive segments, which are then sequentially input into the spike layer for temporal feature extraction and encoding, and the output of the spike layer is connected in the length dimension of the EEG signal. The output of the spike layer is then sent to the channel attention module to capture the channel relationship of the EEG signal.

[0026] The pulse layer of both the EEG branch and the audio branch uses LIF neurons. The membrane potential V of the lth layer neuron at timestamp t is t,l , expressed as formula (3) to formula (6):

[0027] V t,l =H t,l +Z t,l-1 (3)

[0028] z t,l =f(V t-1,l -g) (4)

[0029] H t,l =(αV t-1,l )(1-Z t,l-1 ) (5)

[0030] Z t,0 =E t (6)

[0031] Where f(·) is the step function; g is the membrane potential threshold; α is the leakage factor of the LIF neuron; E t is the tth segment of the EEG decision window or speech envelope decision window; Z t,l represents the output pulse of layer l at timestamp t; H t,l It indicates that the membrane potential of layer l at timestamp t is updated according to the leakage factor and the output pulse.

[0032] The channel attention module of the EEG branch is to capture the relationship between EEG channels and adaptively assign different weights to the channels. The expression is:

[0033] x′=x·σ(W2·ReLU(W1·AvgPoo12d(x))) (7)

[0034] x represents the input of the channel attention module, x′ represents the output of the channel attention module, σ represents the Sigmoid activation layer, W1 and W2 are the weight matrices of the fully connected layer, ReLU(·) represents the relu activation function, and AvgPool2d(·) represents the global average pooling.

[0035] The audio branch contains a pulse layer, which divides the input speech envelope into K consecutive segments aligned with the EEG signal, and then sequentially inputs the pulse layer for sequence time feature extraction and encoding, and splices the output of the pulse layer on the length dimension of the speech envelope.

[0036] Step 4: Classification based on classifier

[0037] The outputs of the two audio branches are connected with the outputs of the EEG branches in the channel dimension to obtain features M1 and M2, and then M1 and M2 are flattened into two-dimensional tensors and input into the feature fusion module to obtain audio and EEG fusion features T1 and T2. The fusion features T1 and T2 are concatenated in the channel dimension and passed through a fully connected layer and softmax activation function to obtain the classification result. The expression of the feature fusion module is as follows:

[0038] Y=MaxPool(ReLU(Conu2d(X))) (8)

[0039] Conv2d represents a convolution operation, X is the input data, Y is the output data, and ReLU represents the activation function applied to the result of the convolution operation. MaxPool represents an adaptive maximum pooling operation, which adjusts the activated feature map to the specified output size.

[0040] The present invention uses a threshold calculation module to provide a threshold for the pulse layer, and uses the EEG branch and the audio branch to process the time-synchronized EEG signals and audio respectively. The EEG branch uses the pulse layer to extract time features and encodes them, in which a channel attention module for EEG is designed; the audio branch uses the pulse layer to extract features and encode the two input audios respectively. Then, the feature fusion module is used to extract the fusion features of EEG and the two audios. Finally, the fully connected layer and softmax are used to complete the classification. It should be noted that the EEG branch and the audio branch are a cyclic structure, and the input EEG and audio need to be divided into K continuous segments, and then enter the EEG branch and the audio branch in sequence to perform the above steps.

[0041] Beneficial effects of the present invention: The present invention introduces pulse neural networks into the auditory attention detection task, simulates the information encoding mechanism of the human brain, solves the problems of weak interpretability and poor real-time performance of the current auditory attention detection model, and achieves high accuracy under low latency and short decision windows. Compared with other methods, the method proposed by the present invention has better generalization and stable detection results on different subjects. In addition, the pulse neural network can greatly reduce the energy consumption of the model when used in conjunction with special neural hardware, which is helpful for hardware development and provides an effective method for developing brain-controlled hearing aids. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is the network structure diagram of this method. DETAILED DESCRIPTION

[0043] The present invention is further described in detail below in conjunction with specific implementation modes, but the present invention is not limited to the specific implementation modes.

[0044] An auditory attention detection method based on a spiking neural network performs data preprocessing, network model training and testing.

[0045] The KUL data set widely used in this task was used for the experiment. The data set has synchronized EEG signals and audio and is tested on multiple subjects, which meets the requirements of the present invention. The preprocessing of EEG and audio data is completed on Matlab: the EEG data is high-pass filtered at 0.5Hz and downsampled to 128Hz. What needs to be completed is to re-reference the EEG data based on the Cz channel of the EEG, and then perform bandpass filtering. The audio is downsampled to 12kHz, and then filtered using the Gammatone filter bank, and the filtered signal is full-wave rectified and power-law compressed, averaged between all filters and then downsampled to 128Hz, and then bandpass filtered using a 2nd-order Butterworth filter to obtain a speech envelope. Before the EEG signal and speech envelope are sent to the network, the mean and variance of the EEG data of each subject need to be calculated, and the EEG data need to be standardized. In order to make the data more suitable for the input of the impulse layer, both the EEG data and the speech envelope are normalized. In order to verify the detection results of this invention in a shorter time window, the EEG signal and speech envelope are divided into continuous decision windows. The decision window can take a value within 2s, without data overlap, and always keep the synchronization between the data.

[0046] The proposed network model is implemented in PyTorch, and cross entropy is used as the loss function. The threshold calculation module is not enabled during the initial N epochs of training. The threshold is first set to a as the guide for the initial threshold for training. After training N epochs, the threshold calculation module is enabled to continue training for M epochs. In the last b epochs of training the threshold calculation model, the thresholds calculated by the threshold calculation module are recorded and their mean is calculated. The mean is used as the fixed threshold for subsequent model training, 0≤N≤100, 0≤M≤100. Experiments show that this strategy of first using a as the threshold guide and then obtaining the threshold mean after a certain period of training as the fixed threshold for subsequent training shows good accuracy. The model is trained using a stochastic gradient descent (SGD) optimizer with a momentum of 0.9. The batch size of the model is 16, the initial learning rate is 0.001, and the leakage factor of the pulse layer is set to 0.2.

[0047] The following is a further explanation of the network model parameters: The EEG signal in the data set is 64 channels. In the preprocessing, the EEG signal has been resampled to 128Hz and the EEG signal has been cut into a 2s decision window for subsequent analysis, that is, each window contains 128×2=256 sampling points. The EEG signal passes through the threshold calculation module to obtain the pulse neural network threshold. The threshold calculation module expression is as follows:

[0048] u=Concat(Mean(E),Std(E)) (1)

[0049] g=Avg(σ(BN(Con(u)))) (2)

[0050] Mean means calculating the mean, Std means calculating the standard deviation, Concat means connecting in the channel dimension, Conv means one-dimensional convolution layer, BN means batch normalization layer, σ means Sigmoid activation layer, Avg means averaging, and g means spike neural network threshold. Among them, the convolution kernel size of the Conv convolution layer is 2, and the number of input / output channels is 1; the BN layer is one-dimensional batch normalization, and the number of channels is 1.

[0051] After that, the preprocessed EEG signal and audio are input into the EEG branch and audio branch respectively, and the pulse neural network threshold g calculated in the previous step is used for calculation in the EEG branch and audio branch. The EEG branch contains a pulse layer and a channel attention module. The input EEG signal is divided into K continuous segments, which are then input into the pulse layer in sequence, and the output of the pulse layer is connected in the length dimension of the EEG signal. The audio branch has only one pulse layer. The pulse layers of the EEG branch and the audio branch both use LIF neurons. At timestamp t, the membrane potential V of the lth layer neuron is t,l , expressed as formula (3) to formula (6):

[0052] V t,l =H t,l +Z t,l-1 (3)

[0053] Z t,l =f(V t-1,l -g) (4)

[0054] H t,l =(αV t-1,l )(1-Z t,l-1 ) (5)

[0055] Z t,0 =E t (6)

[0056] Where f(·) is the step function; g is the membrane potential threshold; α is the leakage factor of the LIF neuron; E t is the tth segment of the EEG decision window or speech envelope decision window; H t,l , Z t,l They represent the membrane potential and output pulse of the lth layer at time stamp t, respectively. In this embodiment, the input EEG signal or audio is divided into segments each having 4 sampling points.

[0057] The channel attention module expression of the EEG branch is:

[0058] x′=x·σ(W2·ReLU(W1·AvgPool2d(x))) (7)

[0059] x represents the input of the channel attention module, x′ represents the output of the channel attention module, σ represents the Sigmoid activation layer, W1 and W2 are the weight matrices of the fully connected layer, ReLU(·) represents the relu activation function, and AvgPool2d(·) represents the global average pooling. Among them, AvgPool2d(·) is an adaptive two-dimensional average pooling operation, which adjusts the height and width of the tensor to 1×1; the first linear layer input is 64-dimensional and the output is 4-dimensional, and the second linear layer input is 4-dimensional and the output is 64-dimensional.

[0060] Finally, based on the classifier, the outputs of the two audio branches are connected with the outputs of the EEG branch in the channel dimension to obtain features M1 and M2. Then M1 and M2 are input into the feature fusion module to obtain the fusion features T1 and T2 of audio and EEG. The expression of the feature fusion module is as follows:

[0061] Y=MaxPool(ReLU(Conv2d(X))) (8)

[0062] Conv2d represents a convolution operation, X is the input data, Y is the output data, and ReLU represents the activation function, which is applied to the result of the convolution operation. MaxPool represents an adaptive maximum pooling operation, which adjusts the activated feature map to the specified output size. Among them, the convolution kernel size of Conv2d is 65×9, the number of input channels is 1, and the number of output channels is 16; MaxPool is an adaptive two-dimensional maximum pooling operation, which adjusts the height and width of the tensor to 1×2, and 2 comes from the decision window of 2s.

[0063] The fused features T1 and T2 are flattened into a two-dimensional tensor and then concatenated in the channel dimension and passed through a fully connected layer and a softmax activation function to obtain the classification result. The expression is as follows:

[0064] T x_reshaped = reshape(T x , shape=(batch_size,-1)) (9)

[0065] R = Softmax(W3·T reshaped ) (10)

[0066] R represents the final classification result, T x Indicates T1 or T2, T x_reshaped Represents the fused features after adjustment to a two-dimensional tensor, T reshaped Represents the fusion feature T 1_reshape , T 2_reshapedThe data after concatenation in the channel dimension, Softmax represents the softmax activation function, and W3 represents the weight matrix of the fully connected layer. The input of the fully connected layer is 48-dimensional, the output is 2-dimensional, and batch_size is the batch size 16 mentioned above.

Claims

1. An auditory attention detection method based on pulse inspiration, characterized in that: The EEG signal and two audios collected synchronously are preprocessed, and the pulse neural network threshold is first calculated through the EEG signal, and then input into the EEG branch and the audio branch respectively; The EEG branch first extracts the temporal features through the pulse layer and completes the encoding, and then applies the channel attention module to capture the channel relationship of the EEG signal; there is an audio branch that processes two audios separately, and the audio extracts the sequence temporal features through the pulse layer and completes the encoding; The output of the EEG branch is concatenated with the output of the two audio branches in the channel dimension, and then the concatenation results are respectively sent to two feature fusion modules to obtain the fusion features between the EEG and the two audios. The fully connected layer maps the fusion features to the same output dimension as the number of categories, and finally the classification result is obtained through the softmax function. The specific steps are as follows: Step 1: Preprocessing of EEG signals and audio The EEG signal was high-pass filtered with a cutoff frequency of 0.5 Hz, and the EEG signal was re-referenced to the EEG Cz channel, downsampled to 128 Hz, and band-pass filtered; The audio is processed using an auditory filter with power-law compression to obtain the speech envelope, and finally downsampled to 128 Hz and bandpass filtered; Standardize and normalize the EEG signal and normalize the speech envelope; Use sliding windows to segment EEG signals and speech envelopes into continuous segments, and always keep the synchronization between EEG signals and audio; Step 2: Calculate the threshold of the spiking neural network based on the EEG signal The threshold calculation module first calculates the mean and standard deviation of each channel of the input EEG signal E, then merges the mean and standard deviation together to obtain a new tensor u, which is then input into the convolution layer and then batch normalized. The Sigmoid function maps the value to the range of (0, 1), and then calculates the average value to obtain the spike neural network threshold g, which is expressed as follows: u=Concat(Mean(E),Std(E))(1) g = Avg(σ(BN(Conv(u))))(2) Among them, Mean means calculating the mean, Std means calculating the standard deviation, Concat means connecting in the channel dimension, Conv means one-dimensional convolution layer, BN means batch normalization layer, σ means Sigmoid activation layer, Avg means averaging, and g means the spike neural network threshold; Step 3: Input the preprocessed EEG signal and audio into the EEG branch and audio branch respectively The EEG branch is based on the Spike-CNN architecture and includes a spike layer and a channel attention module. The input EEG signal is divided into N consecutive segments, which are then sequentially input into the spike layer for temporal feature extraction and encoding. The output of the spike layer is connected in the length dimension of the EEG signal. The output of the spike layer is then sent to the channel attention module to capture the channel relationship of the EEG signal. The pulse layer of both the EEG branch and the audio branch uses LIF neurons. At timestamp t, the membrane potential V t,l , expressed as formula (3) to formula (6): In t,l =H t,l +Z t,l-1 (3) Z t,l =f(V t-1,l -g) (4) H t,l =(αV t-1,l )(1-Z t,l-1 ) (5) With t,0 =E t (6) Where f(·) is the step function; g is the membrane potential threshold; α is the leakage factor of the LIF neuron; E t is the tth segment of the EEG decision window or speech envelope decision window; H t,l Indicates that at timestamp t, the layer membrane potential is updated according to the decay factor and the firing pulse; Z t,l represents the output pulse of the,th layer at timestamp t; The channel attention module expression of the EEG branch is: x′=x·σ(W2·ReLU(W1·AvgPool2d(x))) (7) x represents the input of the channel attention module, x′ represents the output of the channel attention module, σ represents the Sigmoid activation layer, W1 and W2 are the weight matrices of the fully connected layer, ReLU(·) represents the relu activation function, and AvgPool2d(·) represents the global average pooling; The audio branch contains a pulse layer; the input speech envelope is divided into N consecutive segments aligned with the EEG signal, and then sequentially input into the pulse layer for sequence time feature extraction and encoding, and the output of the pulse layer is spliced ​​in the length dimension of the speech envelope; Step 4: Classification based on classifier The outputs of the two audio branches are connected with the outputs of the EEG branch in the channel dimension to obtain features M1 and M2, and then M1 and M2 are flattened into two-dimensional tensors and input into the feature fusion module to obtain the fusion features T1 and T2 of audio and EEG. After that, a fully connected layer and a softmax activation function are used to obtain the classification result. The mathematical expression of the feature fusion module is as follows: Y=MaxPool(ReLU(Conv2d(X))) (8) Conv2d represents a convolution operation, X is the input data, and Y is the output data; ReLU represents an activation function, which is applied to the result of the convolution operation; MaxPool represents an adaptive maximum pooling operation, which adjusts the activated feature map to the specified output size.

2. The method for detecting auditory attention based on pulse elicitation according to claim 1, characterized in that: The threshold calculation module is not enabled during the initial N epochs of training. The threshold is first set to a as a guide for the initial threshold for training. After training N epochs, the threshold calculation module is enabled to continue training M epochs. In the last b epochs of training the threshold calculation model, the thresholds calculated by the threshold calculation module are recorded and their mean is calculated. The mean is used as the fixed threshold for subsequent model training, 0≤N≤100, 0≤M≤100.

Citation Information

Patent Citations

  • Multi-modal emotion recognition method and system for accompanying robot

    CN113947127A

  • Selection and configuration of an automated robotic process

    US20210356941A1

Cited By

  • Auditory attention detection method and device, computer equipment and readable storage medium

    CN119564208A

  • Auditory attention detection method, device, computer equipment and readable storage medium

    CN119564208B

  • Space-time joint spiking neural network electroencephalogram signal identification method and system

    CN122163230A