A method for detecting phonation of EEG signals based on convolutional temporal networks
By combining convolutional timing networks with convolutional modules and long and short-term memory modules in vocalization detection, the problems of deep network complexity and information loss in the existing technology are solved, and efficient sound detection of EEG signals is realized, which improves detection accuracy And an explanation is provided.
Patent Information
- Application Number
- CN202311672981.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-07
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2043-12-07
AI Technical Summary
When the existing vocal detection methods process EEG signals, the deep network structure is complex and the parameters are large, which is difficult to explain; a single neural network does not model the time information, and the principal component analysis cannot use category information, resulting in loss during dimensionality reduction and classification related to classification. information.
Using a method based on convolutional timing network, combining the convolution module and the long and short-term memory module, the convolution module performs feature extraction and dimensionality reduction of the EEG signal, and the long and short-term memory module performs vocalization state classification, using Hilbert transform and threshold value Segmentation of the audio data for envelope analysis and annotation, and a convolutional timing network is constructed for training to achieve vocal detection.
Improve the effect of vocal detection, achieve performance that is better than commonly used vocal detectors, and improve the uninterpretation of neural networks, and provides a biologically interpretable detector, which can better extract the space and Frequency information.
Smart Images

Figure CN117688372B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of electroencephalogram (EEG) data analysis, and in particular relates to an EEG signal utterance detection method based on a convolutional temporal network. Background Art
[0002] The design of vocalization detector is an important part of language brain-computer interface. Accurately detecting the speaking time of the subject can provide accurate and complete feature signals for the subsequent reconstruction of sound features, which is of great help for the design of two-stage speech feature decoder and the screening of speech feature decoder training set data. At present, there have been many studies on the task of vocalization detection. Among them, the commonly used vocalization detectors include Linear Discriminant Analysis (LDA), Support Vector Machine (SVM) and Long Short-term Memory (LSTM). Among them, deep learning algorithms have excellent performance in vocalization detection tasks due to their strong fitting ability and generalization. In order to solve the problem of high dimensionality and large information redundancy of neural signals, existing methods mostly use principal component analysis (PCA) to reduce the dimensionality of neural signals.
[0003] Although there are many algorithms in this area, there are still some problems: the complex structure and large number of parameters of deep networks make their mechanisms difficult to explain well; a single neural network is only good at modeling a single dimension of data. For example, the LSTM network can model time information, but its ability to model spatial and frequency information in EEG signals is insufficient; in addition, unsupervised dimensionality reduction methods such as principal component analysis are based on the statistical properties of the data and cannot utilize the category information of the data, and information related to classification will be lost during dimensionality reduction. Summary of the invention
[0004] In view of the defects of the above-mentioned prior art, the present invention proposes a method for EEG signal speech detection based on a convolutional timing network. The convolutional timing network includes a convolution module and a long short-term memory module. The convolution module performs feature extraction and dimensionality reduction on the EEG signal, and the long short-term memory module classifies the vocalization state of the neural features after dimensionality reduction. The convolution module can effectively extract the EEG signal frequency band that has a high contribution to the vocalization task, thereby improving the effect of vocalization detection and achieving performance that is superior to the currently commonly used vocalization detectors. At the same time, it effectively improves the unexplainable problems of the existing vocalization detection methods based on neural networks.
[0005] A method for detecting phonation of an electroencephalogram signal based on a convolutional temporal network comprises the following steps:
[0006] (1) Collecting raw EEG data and audio data simultaneously, performing time-frequency analysis on the collected raw EEG data, and using the EEG data after time-frequency analysis as neural features;
[0007] (2) Perform envelope analysis on the collected audio data using Hilbert transform, annotate the audio data after envelope analysis through threshold segmentation, obtain speech features, and align speech features and neural features;
[0008] (3) constructing a convolutional temporal network for utterance detection composed of a convolutional neural module and a bidirectional long short-term memory module, using the neural features aligned in step (2) as input data, and the speech features aligned in step (2) as true value labels to construct a data set, training the convolutional temporal network for utterance detection, and obtaining a trained convolutional temporal utterance detector;
[0009] (4) The EEG signal to be detected is input into the convolutional time series speech detector trained in step (3) after time-frequency analysis to predict the pronunciation state and complete the speech detection of the EEG signal.
[0010] In step (1), the collected raw EEG data is subjected to time-frequency analysis, which specifically includes:
[0011] The signal of each electrode channel in the original EEG data is filtered out of noise using the common-mode median reference, followed by downsampling and filtering. The power spectrum estimation is then used to extract the frequency band features of the filtered signal while retaining the time domain features of the filtered signal. Finally, the time domain features and frequency band features are concatenated.
[0012] In step (2), the collected audio data is subjected to envelope analysis using Hilbert transform, and the audio data after envelope analysis is annotated by threshold segmentation to obtain speech features, which specifically include:
[0013] The envelope information of the audio data is calculated using the Hilbert transform, and then the envelope information is normalized to [0,1]. Finally, the normalized envelope information is labeled using the threshold method, and the audio data is labeled as speaking or not speaking to obtain speech features.
[0014] In step (3), the vocalization detection convolutional timing network is composed of a convolutional module and a bidirectional long short-term memory module. The convolutional module extracts and reduces the dimension of neural features, and the bidirectional long short-term memory module performs vocalization detection based on the features after dimensionality reduction.
[0015] In step (3), the convolution module includes alternating convolutional layers and normalization layers;
[0016] The bidirectional long short-term memory module includes a plurality of bidirectional long short-term memory layers connected in sequence.
[0017] In step (3), the neural features aligned in step (2) are used as input data, and the speech features aligned in step (2) are used as true value labels to construct a data set, and the utterance detection convolutional time series network is trained to obtain a trained convolutional time series utterance detector, which specifically includes:
[0018] (3.1) The aligned neural features are input into the convolution module in batches for extraction and dimensionality reduction;
[0019] (3.2) The reduced-dimensional features are sent to the bidirectional long short-term memory module for temporal feature extraction and classification, and the probability distribution prediction x of each sample speech feature is output. n , x n =[x n,1 ,x n,2 ], x n represents the probability distribution prediction of the nth sample, x n,1 represents the probability of predicting that x belongs to category one, x n,2 Represents the probability of predicting that x belongs to category 2;
[0020] (3.3) Use the normalized exponential function to normalize the probability distribution of the output speech features, and then use the cross entropy loss function to calculate the classification loss of the network. The calculation formula is as follows:
[0021]
[0022]
[0023]
[0024] where x n,c ′ represents the normalized probability distribution, l n represents the cross entropy loss of the nth sample, C represents the total number of categories, and in this task, C = 2, which represents the two states of voice and silence, and w c is the weight of the c-th type loss, y n =[y n,1 ,y n,2 ] is the true value label of the nth sample in the aligned speech feature, is the actual probability distribution of the nth sample, for the true category i of the sample, y n,i =1, for other categories c≠i, y n,c = 0. N represents the batch size, and the sum of the cross entropy losses of all samples in a batch is the final loss function L;
[0025] (3.4) After calculating the classification loss, the network parameters of the sound detection convolutional timing network are updated.
[0026] Specifically, the present invention provides a method for detecting phonation of EEG signals based on a convolutional temporal network, comprising the following steps:
[0027] (1) Perform time-frequency analysis on the collected raw EEG data and use the preprocessed EEG data as neural features;
[0028] (2) Perform envelope analysis on the audio using Hilbert transform, annotate the audio data through threshold segmentation, and align speech features and neural features;
[0029] (3) Construct a convolutional time series network for voice detection composed of a convolutional neural module and a bidirectional long short-term memory module, use the neural features in step (1) as input data, and the audio data annotation as the true value label to construct a data set and train the convolutional time series network;
[0030] (4) The EEG signal to be detected is processed in step (1) and then input into the convolutional time series speech detector trained in step (3) to predict the pronunciation state and realize the speech detection of EEG data.
[0031] The preprocessing process in step (1) includes the following steps:
[0032] 1) Re-reference to filter out noise: Use the common median reference (CMR) to process the raw data. First, calculate the median of all electrode channel signals at the current sampling point t as the common mode noise, and then subtract the common mode noise from the signal of each channel. The calculation formula is as follows:
[0033] X i (t) = X i (t)-Me(X1(t),X2(t),…,X n (t))
[0034] Where n is the number of channels, X i (t) represents the signal of the i-th channel at time t, and Me represents the median calculation.
[0035] 2) Downsampling and low-pass filtering: downsample all channel data to 2kHz, and then perform 500Hz low-pass filtering on all channel data after downsampling.
[0036] 3) Power spectrum estimation: The processed EEG data is segmented by sliding window, and the spectrum graph is calculated for each segment using short-time Fourier transform. The formula is as follows:
[0037]
[0038] P i (t,ω)=log 10 (∥Fi (t,ω)∥ 2 )
[0039] F i (t,ω) is the signal strength of the EEG signal of the ith channel at the frequency ω at time t, g represents the window function, and this method uses a Hamming window with a window length of 200ms and a sliding step of 10ms; X i (t) represents the EEG signal of the ith channel at time t; then i (t,ω) is squared and the logarithm is calculated to obtain the power P of the EEG signal of the i-th channel at the frequency ω at time t i (t,ω).
[0040] 4) Frequency band feature extraction: The spectrum features are divided into 20 frequency band features with a bandwidth of 10 Hz (0 Hz-10 Hz, 10 Hz-20 Hz, ..., 190 Hz-200 Hz) according to the size of the center frequency.
[0041] 5) Time domain feature extraction: In addition to frequency domain features, the dataset also includes 1-dimensional local field potential (LFP) features, which are time domain feature values after bandpass filtering from 0.5 Hz to 5 Hz.
[0042] 6) Use the Z-Store method to standardize the feature data. The calculation formula is as follows:
[0043]
[0044] Where P i (t) is the size of the i-th eigenvalue at time t, μ i and σ i represent the mean and standard deviation of the ith feature respectively.
[0045] In step (2), the audio is annotated. First, the Hilbert transform is used to perform envelope analysis on the collected sound signal. The calculation formula is as follows:
[0046] x a =F -1 (F(x)2U)=x+iy
[0047]
[0048] where x arepresents the analytical signal of the sound signal x, F represents Fourier transform, and U represents the step function. y represents the Hilbert transform of x, and the absolute value of the analytical signal is the envelope information of the sound signal. After calculating the envelope information, the envelope information is normalized to [0,1], and then the threshold method is used to determine whether the voicing state of the current sampling point is speaking or not speaking.
[0049] The convolutional temporal network structure constructed in step (3) includes three convolutional layers with convolution kernels of 64, 32, and 16, a flattening layer, three bidirectional long short-term memory layers, and the final fully connected layer. The convolutional layer is first used to extract features and reduce the dimension of the EEG signal, and then the data format is unified through the flattening layer. Then, the bidirectional long short-term memory layer is introduced to use the information of the previous and next time windows to assist in the judgment of the current time state of the voice, and finally the fully connected layer is used to output the binary classification features of each time point.
[0050] Compared with the prior art, the present invention has the following beneficial effects:
[0051] The present invention develops an EEG signal vocalization detection method based on a convolutional neural network and long short-term memory network (CNN-LSTM) to detect speech from EEG. By adding a convolution module before the bidirectional long short-term memory network, the model can better extract the spatial and frequency information in the EEG signal, improving the detection accuracy of the model; at the same time, the convolution network can extract low-frequency signals that are highly correlated with the vocalization task, which is consistent with the characteristics of neural signal release in the vocalization process in neuroscience, providing an explanation for the excellent performance of the network, and a biologically interpretable EEG signal vocalization detector.
[0052] A convolutional time series sound detector constructed by the present invention has achieved performance superior to that of currently commonly used sound detectors. At the same time, the present invention explains the detection performance of the detector, effectively improving the unexplainable problem of existing sound detection methods based on neural networks. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 It is a schematic diagram of the process of the present invention;
[0054] Figure 2 The process of collecting EEG data and audio data;
[0055] Figure 3 It is a schematic diagram of the structure of the convolutional temporal network;
[0056] Figure 4 Performance comparison of Convolutional Temporal Networks and other sound detectors. DETAILED DESCRIPTION
[0057] In order to describe the present invention more specifically, the technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0058] The present invention provides a method for detecting the sound of an EEG signal based on a convolutional time series network. Figure 1 The following steps are shown:
[0059] (1) Data collection. Figure 2 As shown, an 8×9 ECoG electrode is used to collect the EEG signal of the subject. The subject is codenamed P5. The ECoG electrode is connected to the PMT pigtail adapter through a percutaneous pigtail, and then connected to two 64-channel shunts through a separate anti-touch connector. The signal of the shunt is amplified by the preamplifier and input into the neural signal processor NeuroPort. During the experiment, a microphone was placed about 10 cm in front of the subject's lips to collect the subject's voice signal. After the sound signal is collected, it is amplified using an audio system, and then the sound signal is transmitted to one of the NeuroPorts and synchronized with the neural signal. In the experiment, there is a display 1 meter in front of the subject. According to the prompts on the screen, the subject reads aloud the French vowels or short sentences on the screen, and then the screen is randomly black for 0 to 1 second.
[0060] (2) Preprocess the neural signals and speech data collected in step (1). For neural data, first perform common-mode median reference filtering on the original neural signal; then downsample the sampling rate of the neural signal to 2kHz; then filter the neural signal using a low-pass filter with a cutoff frequency of 500Hz; finally perform sliding window segmentation and use Fourier transform to convert the signal into a spectrum representation, and divide the extracted features into 20 frequency bands with a bandwidth of 10Hz (0-10Hz, 10-20Hz, 20-30Hz, ..., 190-200Hz) according to the center frequency. At the same time, retain the 1-dimensional time domain feature, which is the time domain feature value after 0.5Hz to 5Hz bandpass filtering, downsample the time domain feature to 100Hz and splice it with the frequency domain feature, and finally obtain a signal feature of 8×9×21=1512 dimensions. Finally, perform z-score normalization on the feature.
[0061] For speech data, the energy of the sound signal is calculated to determine whether the subject is speaking. First, the Hilbert transform is used to obtain the analytical signal of the sound signal. The absolute value of the analytical signal is the envelope information of the sound signal. The envelope information is normalized, and the threshold method is used to segment the subject's vocal state. After multiple tests, the threshold size is set to 0.05. Then, according to the time before and after the subject spoke in each trial, 500ms is expanded as a sample. While segmenting the speech data, the neural features need to be segmented in the same way according to the time label. Since the subject speaks most of the time, the final positive and negative sample ratio is 1.85. In order to alleviate the problem of sample imbalance, the subject's resting neural signal is collected as a negative sample to expand the data set. The final positive and negative sample ratio is 1.07.
[0062] (3) Construct a network model, such as Figure 3 As shown in the figure, three convolutional layers with a convolution kernel of 3×3, a step size of 1, and channels of 64, 32, and 16 are used to extract the spatial and frequency features of the EEG signal to achieve dimensionality reduction of the EEG data. Then a flattening layer is connected to change the data dimension. Then, three bidirectional long short-term memory layers are used to extract the time features, and the information of the previous and next time windows is used to assist in the judgment of the current vocalization state. Finally, a fully connected layer is used to output the two-category vocalization features of each sampling point.
[0063] The constructed pronunciation detection network is trained. The input EEG signal size is T×1512, where T is the number of sampling points. After three convolutional layers, the data becomes T×96. After three long short-term memory network layers, the signal dimension becomes, and finally after a fully connected layer, a T×2 sequence classification feature is output. The cross entropy loss function L is calculated for the classification sequence and the T×2 label:
[0064]
[0065]
[0066]
[0067] x n represents the probability distribution prediction of the nth sample, x n =[x n,1 ,x n,2 ], x n,1 represents the probability of predicting that x belongs to category one, x n,2 Represents the probability of predicting that x belongs to category 2; where x n,c ′ represents the normalized probability distribution, l n represents the cross entropy loss of the nth sample, C represents the total number of categories, and in this task, C = 2, which represents the two states of voice and silence, and w cis the weight of the c-th type loss, y n =[y n,1 ,y n,2 ] is the true value label of the nth sample in the aligned speech feature, is the actual probability distribution of the nth sample, for the true category i of the sample, y n,i =1, for other categories c≠i, y n,c = 0. N represents the batch size, and the sum of the cross entropy losses of all samples in a batch is the final loss function L;
[0068] Back propagation, calculate the gradient of the loss function to each layer of network parameters through the chain rule, and use the optimization algorithm to update the parameters of the network, as shown below:
[0069]
[0070] Among them, θ n+1 is the updated value of the parameter, θ n is the current parameter value, η is the learning rate, is the derivative of the loss function with respect to the parameter θ.
[0071] The training step requires multiple iterations, each of which includes a forward propagation and a back propagation, as well as parameter updates. Throughout the process, the network parameters are continuously adjusted to make the network's predicted output closer to the true label in order to minimize the loss function. The training process can set stop conditions, such as reaching a predetermined number of iterations, or the convergence of the loss function. Once the stop condition is met, the training process ends, the network parameters are fixed, and can be used to predict new samples.
[0072] (4) The EEG signal to be detected is processed in step (1) and then input into the voice detector trained in step (3), and the pronunciation status of each sampling point is output to realize voice detection of EEG data.
[0073] According to the above steps, the designed sound detection network achieved 90.5% accuracy and 95.4% recall on the data set of subject P5. Figure 4 As shown in a, the horizontal axis in the figure represents four different evaluation indicators (accuracy, precision, recall and F1 score), and each box plot represents the distribution of all test results of 10-fold cross validation. The diamond block represents the outlier of the box plot. From left to right, the results of LDA, SVM, LSTM and CNN-LSTM are respectively shown. Above the image is the significance test of the detection results of adjacent decoders (paired t-test, Bonferroni correction, **: p<0.01). It can be seen that on the dataset of subject P5, the performance of the detector is significantly higher than that of LDA, SVM and LSTM.
[0074] The LSTM detector in the figure uses PCA to reduce the dimension of EEG features, and then feeds the features into LSTM for classification. The performance of CNN-LSTM is better than LSTM (paired t-test, Bonferroni correction, p<0.01), indicating that CNN can extract spatial and frequency domain features from ECoG signals, and the extracted features are more relevant to the vocalization detection task than the features extracted by PCA. The detection results of LSTM are also significantly better than those of LDA and SVM, which indicates that time series features and nonlinear detectors can provide a wider feature fitting space, thereby obtaining results that are better than those of linear detectors. For subjects S3 and S6, the performance comparison of the four detectors is similar to the results of subject P5. The data sets of subjects S3 and S6 are a set of sEEG data sets publicly released by the Christian Herff team of Maastricht University in the Netherlands.
[0075] Figures d, e, and f are the detection results of random trials of the CNN-LSTM detector. The horizontal axis represents time, the lower vertical axis represents the silent part, and the higher vertical axis represents the speaking part. The straight line represents the prediction result of CNN-LSTM, and the dotted line represents the real speaking label. It can be seen that CNN-LSTM has a high detection accuracy, with only a small amount of advance or delay at the start or end of the speech. In the figure, the CNN-LSTM detector has a high accuracy rate and a high recall rate, which means that CNN-LSTM can not only accurately detect the time interval of the subject's speech, but also detect as many speaking parts as possible. This is of great help to the design of online language brain-computer interface decoders, which means that we can use the detector to detect the interval segment of the subject's speech and only use the neural signal of the subject's speaking part for speech decoding. This can provide more accurate context information for LSTM and improve the decoding of signals with strong temporal correlation such as audio features.
[0076] Finally, the frequency feature extraction effect of the convolution module is analyzed. The test data is input into the network, the output of the convolution module is extracted, and the mutual information is used to calculate the mutual information between the convolution module output and the 21 frequency band neural signals. The correlation between the detector convolution module output and the neural signal is evaluated using mutual information, and then the frequency feature components extracted by the convolution network from the neural signal input are analyzed. The mutual information calculation formula is as follows:
[0077]
[0078] Among them, X represents the neural signal, Y represents the output of the convolution module, and I(X,Y) represents the mutual information between the output of the convolution module and the neural signal in a specific frequency band. p(x) represents the marginal probability distribution of X=x, p(y) represents the marginal probability distribution of Y=y, and p(x,y) represents the joint distribution of x and y. The calculation results show that most of the outputs of the convolution layer have higher mutual information with the 40-60Hz EEG signal. Previous studies have shown that the γ frequency band in the 40-60Hz range of ECoG EEG signals has a high correlation with the envelope of the sound signal, which shows that the detector extracts this part of low-frequency features that are more relevant to the occurrence detection task, which explains the high performance of the detector from the perspective of the neural signal frequency band. At the same time, PCA is used to reduce the dimension of the neural signal, and the mutual information between the PCA output and the neural signal is calculated. It is found that the features extracted by PCA have no obvious preference for frequency bands, and each principal component extracts information in each frequency band. This is mainly because PCA is an unsupervised algorithm, and the extracted features are not necessarily related to the vocal task. The detector designed according to the present invention has a complete neural network because the modules are connected to form a complete neural network. When the network parameters are updated in the training phase, the task-related frequency band features will be given higher weights according to the loss function.
Claims
1. A method for detecting phonation of EEG signals based on convolutional temporal networks, characterized in that: The following steps are involved: (1) Collecting raw EEG data and audio data simultaneously, performing time-frequency analysis on the collected raw EEG data, and using the EEG data after time-frequency analysis as neural features; (2) Use Hilbert transform to perform envelope analysis on the collected audio data, annotate the audio data after envelope analysis through threshold segmentation, obtain speech features, and align speech features and neural features; The collected audio data is analyzed using the Hilbert transform, and the audio data after envelope analysis is annotated through threshold segmentation to obtain speech features, including: The envelope information of the audio data is calculated using the Hilbert transform, and then the envelope information is normalized to [0, 1]. Finally, the normalized envelope information is labeled using the threshold method to mark the audio data as speaking or not speaking, thereby obtaining speech features. (3) constructing a convolutional temporal network for utterance detection composed of a convolutional neural module and a bidirectional long short-term memory module, using the neural features aligned in step (2) as input data, and the speech features aligned in step (2) as true value labels to construct a data set, and training the convolutional temporal network for utterance detection to obtain a trained convolutional temporal utterance detector; The vocalization detection convolutional temporal network is composed of a convolution module and a bidirectional long short-term memory module. The convolution module extracts and reduces the dimension of neural features, and the bidirectional long short-term memory module performs vocalization detection based on the features after dimensionality reduction. The convolution module includes convolution layers and normalization layers alternating in sequence; The bidirectional long short-term memory module includes a plurality of bidirectional long short-term memory layers connected in sequence; Three convolutional layers with a 3×3 kernel, a stride of 1, and 64, 32, and 16 channels are used to extract the spatial and frequency features of the EEG signal and reduce the dimensionality of the EEG data. A flattening layer is then connected to change the data dimension. Three bidirectional long-short-term memory layers are then used to extract the temporal features. Information from the previous and next time windows is used to assist in determining the current vocalization state. Finally, a fully connected layer is used to output the binary classification features of the vocalization at each sampling point. (4) The EEG signal to be detected is input into the convolutional temporal speech detector trained in step (3) after time-frequency analysis to predict the pronunciation state and complete the speech detection of the EEG signal.
2. The method for detecting utterances of EEG signals based on a convolutional temporal network according to claim 1, wherein: In step (1), the collected raw EEG data is subjected to time-frequency analysis, specifically including: The signal of each electrode channel in the original EEG data is filtered out of noise using the common-mode median reference, followed by downsampling and filtering. The frequency band features of the filtered signal are then extracted using power spectrum estimation while retaining the time domain features in the filtered signal. Finally, the time domain features and frequency band features are spliced together.
3. The method for detecting utterances of EEG signals based on a convolutional temporal network according to claim 1, wherein: In step (3), the neural features aligned in step (2) are used as input data, and the speech features aligned in step (2) are used as true value labels to construct a data set, and the utterance detection convolutional time series network is trained to obtain a trained convolutional time series utterance detector, which specifically includes: (3.1) The aligned neural features are input into the convolution module in batches for extraction and dimensionality reduction; (3.2) The reduced-dimensional features are fed into the bidirectional long-short-term memory module for temporal feature extraction and classification, and the probability distribution prediction x of each sample speech feature is output. n , x n =[x n,1 , x n,2 ], x n Represents the probability distribution prediction of the nth sample, x n,1 Indicates the probability of predicting x to belong to category one, x n,2 Indicates the probability of predicting x to belong to category 2; (3.3) Use the normalized exponential function to normalize the probability distribution of the output speech features, and then use the cross entropy loss function to calculate the classification loss of the network. The calculation formula is as follows: where x n,c ′ represents the normalized probability distribution, l n represents the cross entropy loss of the nth sample, C represents the total number of categories, and in this task, C = 2, indicating the two states of voice and silence, w c is the weight of the c-th category loss, y n =[y n,1 ,y n,2 ] is the true value label of the nth sample in the aligned speech features, N represents the batch size, and the sum of the cross entropy losses of all samples in a batch is the final loss function L; (3.4) After calculating the classification loss, the network parameters of the convolutional temporal network for voice detection are updated.
Citation Information
Patent Citations
Multi-task epilepsy electroencephalogram automatic detection and prediction system
CN116304575A
Electroencephalogram signal voice decoding method based on generative adversarial network
CN116364096A
