An audio feature extraction method based on brain neural model

Abstract

The present invention discloses an audio feature extraction method based on a brain neural model, comprising the following steps: obtaining speech signal data, preprocessing the speech signal, and dividing it into N speech segments; extracting acoustic features representing the emotional state from each speech segment, normalizing the acoustic features and synthesizing a multi-channel speech spectrogram; inputting the multi-channel speech spectrogram into an RBA-FE model, outputting a BDI score, predicting and classifying the degree of depression based on the BDI score, adjusting the parameters of each network layer in the RBA-FE model based on the difference between the predicted classification and the actual classification, and obtaining an optimal RBA-FE model for depression detection. The present invention simulates the cell selectivity in the auditory cortex of the human brain and adopts an adaptive activation mechanism to solve the overfitting and noise resistance of LSTM, thereby achieving accurate recognition of depressive speech under different noise interferences.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an audio feature extraction method based on a brain nerve model. Background Art

[0002] With the advent of deep learning, a large number of neural network architectures have been developed for audio data analysis. For example, WavDepressionNet analyzes raw speech signals and predicts depression severity using representation and evaluation blocks. Similarly, SpeechFormer++ classifies depression using paralinguistic features. It employs a unit encoder to model inter-speech information and fuses blocks to generate features of varying granularity. Other models, such as Bidirectional Long Short-Term Memory (Bi-LSTM) and Time-Distributed Convolutional Neural Networks (CNN), focus on capturing temporal dependencies, with CNN capturing pre-sound features and Bi-LSTM focusing on temporal relationships. However, most of these learning models must be trained on curated datasets, limiting their applicability and robustness. Furthermore, these models use fixed-threshold activation functions (such as sigmoid), which are prone to overfitting in noisy environments. They also lack adaptive neuron mechanisms, preventing dynamic feature weighting, leading to noise amplification. This limitation is particularly severe in home environments, as recording devices such as mobile phones and computers are often non-professional and susceptible to ambient noise.

[0003] For home-based depression diagnosis, environmental noise can significantly degrade the quality of audio collection. Therefore, there is an urgent need for a highly noise-robust model specifically for home-based depression diagnosis. According to the United Nations Environment Programme's 2022 Frontiers Report, noise levels in most cities worldwide exceed acceptable limits, with low-rise residents frequently exposed to vehicle and human-induced noise. Due to quality issues, audio collection devices such as microphones also introduce electronic noise. It is recognized that environmental noise and reverberation can degrade the performance of depression diagnosis models. Deep Residual Shrinkage Networks (DRSNs) have been used to address noise issues in time series anomaly detection. However, these methods cannot be directly applied to noisy audio samples. When applied directly to audio data, thresholding can filter out key features of depression (such as fundamental frequency variations), and the lack of a mechanism for accumulating temporal features makes it impossible to capture the dynamic patterns of speech signals.

[0004] The LIF-based SNN model has shown satisfactory performance in processing ambient noise in audio recognition and retains some of the temporal features of the audio. However, a limitation of the standard LIF model is its rigid reset threshold, which cannot adapt to the heterogeneity of individual signal points in the time domain. Summary of the Invention

[0005] In order to overcome the above-mentioned shortcomings and deficiencies of the prior art, an object of the present invention is to provide an audio feature extraction method based on a brain neural model.

[0006] To address these challenges, we developed a robust brain-inspired audio feature extractor (RBA-FE) model for audio diagnosis of depression. This RBA-FE model integrates a temporal convolutional neural network, multi-head attention, and a bidirectional long short-term memory network, leveraging the unique strengths of each element to address complex diagnostic challenges. We further proposed an adaptive rate smooth leaky integrate-and-fire (ARSLIF) neuron model, which mimics the "cell selectivity" of the auditory cortex and addresses gate overfitting and noise sensitivity in LSTMs by using adaptive thresholds.

[0007] The purpose of the present invention is achieved through the following technical solutions:

[0008] An audio feature extraction method based on a brain neural model, comprising:

[0009] Acquire speech signal data, pre-process the speech signal data, and divide it into N speech segments;

[0010] Extracting acoustic features representing the emotional state from each speech segment, the acoustic features including pitch, jitter, CQL cepstral, Mel-frequency cepstral coefficients, and first-order and second-order differential entropies of Mel-frequency cepstral coefficients, and synthesizing a multi-channel speech spectrogram after normalizing the acoustic features;

[0011] The multi-channel speech spectrogram is input into the RBA-FE model, and the BDI score is output. The depression level is predicted and classified according to the BDI score. According to the difference between the predicted classification and the actual classification, the parameters of each network layer in the RBA-FE model are adjusted to obtain the optimal RBA-FE model for depression level detection.

[0012] Furthermore, the RBA-FE model adopts a hierarchical structure, which includes a temporal convolution layer, a multi-head attention layer, and a bidirectional long short-term memory layer in sequence;

[0013] Temporal convolution layer, used to extract the spatiotemporal features of multi-channel speech spectrograms;

[0014] A multi-head attention layer is used to perform weighted processing on spatiotemporal features and output a global feature representation of the speech spectrum. The multi-head attention layer is added with a 75% dropout layer.

[0015] A bidirectional long short-term memory layer is embedded in the ARSLIF neuron model. It captures past and future contextual information in the global feature representation through independent forward LSTM units and backward LSTM units, extracts the temporal dynamics of complex emotions in speech signals, and achieves a comprehensive understanding of speech sequences.

[0016] Furthermore, the ARSLIF neuron model is obtained by introducing adaptive threshold adjustment into the standard LIF model.

[0017] Furthermore, the ARSLIF neuron model is embedded in the bidirectional long short-term memory layer through adaptive threshold adjustment and membrane potential leakage mechanism to replace the Sigmoid Activation function.

[0018] Furthermore, the membrane potential change mechanism of the ARSLIF neuron model is as follows:

[0019] (1)

[0020] (2)

[0021] in, V mem Initial membrane potential, V th Membrane potential threshold, τ mem Membrane potential constant, V mem_new is the updated membrane potential;

[0022] In the ARSLIF neuron model, the membrane potential changes with the input signal. Inputs When the threshold is exceeded, the neuron fires a pulse signal and the membrane potential is reset, as follows:

[0023] (3)

[0024] (4)

[0025] Where r(t) is the average emissivity, f adap (t) is the adaptive target emissivity, and the expected emissivity is defined as f , the active firing rate of the ARSLIF neuron model f active and the resting firing rate of the ARSLIF neuron model f rest , irepresents the number of the neuron, R represents the membrane resistance constant, n is the number of neurons, t is the time, V th is the membrane potential threshold.

[0026] Furthermore, the adaptive threshold adjustment is to adaptively adjust the membrane potential threshold by calculating the error with the expected firing rate. V th , specifically:

[0027] ARSLIF neuron model V th Rather than being a fixed constant, it is adaptively adjusted by the error from the desired emissivity:

[0028] (5)

[0029] (6)

[0030] in, τ adapt is the time constant, α and β Adjustable hyperparameters, r e is the rate error, V th_new is the new threshold after update.

[0031] Furthermore, the temporal convolution layer includes multiple cascaded convolution layers and pooling layers.

[0032] Furthermore, the specific process of the temporal convolution layer is as follows:

[0033] The acoustic features are convolved with a 7×7 convolution kernel to extract the comprehensive features of the speech features by expanding the receptive field. After batch normalization, ReLU activation and average pooling, they are input into the residual network with a 3×3 convolution kernel to further extract the detailed features of the speech signal.

[0034] Furthermore, the mean absolute error and mean square error are used to evaluate the difference between the predicted classification and the actual classification.

[0035] Furthermore, the multi-head attention layer includes eight attention "heads" arranged in parallel to learn different acoustic features.

[0036] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0037] (1) The model architecture of the present invention is superior, enhancing the audio feature extraction capability. The proposed RBA-FE module adopts a hierarchical structure. In addition, conventional LSTM or CNN can only extract local temporal features and have difficulty capturing long-term dependencies. The present invention adds multi-head self-attention after T-CNN, which can globally assign weights to multiple time points of the input audio, emphasizing key information and ignoring irrelevant noise.

[0038] Combined with the ARSLIF neuron model, the attention mechanism not only focuses on content but also adapts to changes in ambient noise, enhancing the model's generalization across diverse contexts. This mechanism improves the ability to model the temporal correlations of audio signals, enabling the model to more accurately identify speech characteristics of patients with depression (such as pitch variations and pause patterns).

[0039] Traditional CNNs focus only on local, short-term features, while LSTMs suffer from the vanishing gradient problem when modeling long-term sequences. This paper employs a temporal convolutional neural network (T-CNN) to reduce redundant information during feature extraction and improve computational efficiency. Furthermore, a bidirectional LSTM (Bi-LSTM) is incorporated to capture emotional features over a longer time span, leveraging bidirectional information flow to enhance the model's emotion recognition capabilities. This allows the model to simultaneously learn short-term and long-term speech patterns associated with depression, improving classification accuracy and robustness.

[0040] (2) The ARSLIF neuron model has strong noise resistance and good stability;

[0041] (3) The present invention has strong generalization ability and has good implementation effects on multiple data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is the overall structural diagram of the present invention;

[0043] Figure 2 Schematic diagram of the RBA-FE model structure of the present invention;

[0044] Figure 3 Schematic diagram of the self-adaptation and threshold adjustment principle of the ARSLIF neuron model of the present invention;

[0045] FIG4(a) to FIG4(f) are schematic diagrams showing the performance comparisons of the ARSLIF neuron model of the present invention and other activation functions under three noise environments;

[0046] Figure 5 Figure 2 is a schematic diagram of the ROC curves of different models on the MODMA dataset. DETAILED DESCRIPTION

[0047] The present invention will be further described in detail below with reference to the examples, but the embodiments of the present invention are not limited thereto.

[0048] Example 1

[0049] like Figure 1 As shown, an audio feature extraction method based on a brain neural model includes a feature extractor and an RBA-FE model. The original speech sample data is preprocessed and acoustic features are extracted by the feature extractor, and then input into the RBA-FE model for depression level detection and identification.

[0050] The specific steps are as follows:

[0051] S1 obtains voice signal data, preprocesses the voice signal, and divides it into N voice segments;

[0052] The dataset used in Example 1 is the AVEC 2014 depression dataset. The AVEC 2014 depression dataset consists of 300 video samples recorded by subjects in two tasks, including text reading and question answering. The length of the videos ranges from 6 seconds to 4 minutes and is divided into a training set, a development set, and a test set. Each of the three sets includes 100 audio samples.

[0053] The pretreatment mainly includes:

[0054] First, the VideoFileClip library is used to extract speech information from the video at 22,050 Hz, and a -30 dB clipping filter is applied to remove irrelevant information. The sampling frequency is 1024 Hz, the window length is 1024, and there is no overlap. The speech signal is then segmented into 32-frame windows. In Example 1, the samples are segmented into 1.5 millisecond speech segments. Samples shorter than the specified length are repeated to reach the segment length, and the excess is discarded.

[0055] S2 uses the librosa library to extract acoustic features from each speech segment to characterize the emotional state. These acoustic features include pitch, jitter, CQL cepstral, Mel-frequency cepstral coefficients, and first- and second-order differential entropies of Mel-frequency cepstral coefficients. The acoustic features are normalized and synthesized into a multi-channel speech spectrogram of [batch_size, 128, 64, 6].

[0056] Mel-frequency cepstral coefficients (MFCCs) contain important spectral information of audio, and their first- and second-order differential entropies reveal the dynamic information of speech signals. Pitch is closely related to the severity of depression. Jitter is used to quantify irregular changes in speech frequency associated with severe depression. CQL cepstral improvement analyzes harmonic structure to reveal the impact of emotions on speech. The acoustic features are normalized to synthesize a multi-channel speech spectrogram.

[0057] S3 Input the multi-channel speech spectrogram into the robust brain-inspired audio feature extraction model, namely the RBA-FE model, to perform feature extraction and depression level prediction. According to the difference between the predicted classification and the actual classification, the RBA-FE model parameters are adjusted. The RBA-FE model parameters include the parameters of each network layer, specifically the convolution kernel of the temporal convolution layer, the number of attention heads in the multi-head attention layer, the target firing rate of the ARSLIF in the bidirectional long short-term memory layer. f , initial membrane potential threshold V th_0 , ARSLIF active firing rate f active , ARSLIF resting firing rate f rest , time constant τ adapt The optimal RBA-FE model is obtained by adjusting the hyperparameters α and β and the training parameters of the model (learning rate and batch size) to detect the degree of depression. The RBA-FE model outputs a BDI score, and the depression degree is predicted and classified according to the BDI score.

[0058] This method uses the Beck Depression Inventory (BDI) score to assess the subject's depression level. The data is organized according to the subject ID. To reduce the potential fluctuation of BDI scores, the average value of each subject in the training set and test set is calculated to reduce the inaccuracy of the label.

[0059] This embodiment uses MAE (mean absolute error) and MSE (mean square error) to evaluate the error between the predicted value and the actual value.

[0060] The RBA-FE model is inspired by the human brain's audio information processing mechanism and adopts a hierarchical structure that integrates a temporal convolution layer, a multi-head attention layer, and a bidirectional long short-term memory layer. Figure 2 shown.

[0061] Furthermore, research on the auditory pathway has shown that sound processing in the brain proceeds through multiple hierarchical stages, from basic acoustic feature extraction in early auditory areas, such as the cochlear nucleus and superior olivary complex, to complex temporal processing in the cortex. These stages are shown to be associated with specific layers of deep neural networks (DNNs) for audio processing, supporting the use of a hierarchical architecture for robust feature extraction. Inspired by this brain architecture, the RBA-FE model utilizes a T-CNN for early feature processing, multi-head attention to selectively focus on relevant parts of the audio spectrogram, a Bi-LSTM to integrate information across frames, and either a softmax layer for classification or a fully connected layer for regression.

[0062] The temporal convolutional layer is used to analyze and extract temporal information, namely, spatiotemporal features, from the multi-channel speech spectrogram. The temporal convolutional layer (T-CNN) comprises multiple cascaded convolutional and pooling layers to extract local features of the signal. The six extracted acoustic features are first convolved with a 7×7 convolution kernel to expand the receptive field, extracting comprehensive speech features. After batch normalization, ReLU activation, and average pooling, the layers are then fed into a residual network with a 3×3 kernel to further extract detailed features of the speech signal. The temporal convolutional layer (T-CNN) captures both local and global information of the speech signal through two convolutions, extracting rich temporal features. The output of the T-CNN is concatenated along the temporal dimension, forming the foundation for the subsequent feature analysis network, simulating the function of the primary visual cortex in the human brain.

[0063] The multi-head attention layer is used to weight the spatiotemporal features extracted by the temporal convolutional layer and output a global feature representation of the speech spectrum. This output contains key emotional information and spatiotemporal dependencies. This amplifies the sparse features of the input, accelerating the convergence of the subsequent bidirectional long short-term memory (Bi-LSTM) layer to key features.

[0064] The multi-head attention layer consists of eight attention heads, each of which learns representations of input data in different subspaces (such as pitch, energy, and spectral dynamics), enhancing the model's ability to perceive multi-dimensional information. Their parallel arrangement suppresses noise interference with the single attention mechanism, enhancing the model's robustness. This design of eight attention heads has been experimentally verified to be the optimal performance setting for balancing computational complexity and model expressiveness. A 75% dropout layer is also incorporated into the multi-head attention layer to prevent overfitting, enrich the model's feature interpretation capabilities, and reduce overfitting. The multi-head attention layer mimics the human auditory system's mechanism for selectively focusing on and processing specific audio information.

[0065] The bidirectional long short-term memory layer captures the past and future contextual information in the global feature vector representation through independent forward LSTM units and backward LSTM units, extracts the temporal dynamics of complex emotions in the speech signal, and achieves a comprehensive understanding of the speech sequence. In addition, in response to the ambient noise and reverberation in the home environment, the LSTM unit Sigmoid The activation function is replaced by the ARSLIF neuron model, which is implemented by introducing an adaptive threshold adjustment mechanism into the standard LIF model. It uses adaptive threshold adjustment and smooth leakage mechanisms to simulate the "selective readjustment" of human brain nerve cells, enhancing the model's robustness under noise and significantly improving the accuracy of audio diagnosis of depression.

[0066] The ARSLIF neuron model is inspired by the human brain's audio processing mechanism, and introduces the "cell selective readjustment" mechanism into the LSTM unit. It dynamically filters noise through adaptive threshold adjustment, improves the model's robustness to noise, and enhances key features to improve the accuracy of depressive speech diagnosis. Sigmoid Gate function, first redesigned the gradient propagation mechanism to be compatible with the back-propagation algorithm, solving the timing alignment and information matching problems of the continuous information flow in the LSTM unit and the discrete pulses of the ARSLIF neurons; also designed smooth leakage and parameter constraints to avoid the gradient explosion or vanishing problems caused by the adaptive threshold adjustment mechanism, and increase the stability of the model.

[0067] The working principle of the ARSLIF neuron model is shown in the attached Figure 3 As shown, specifically:

[0068] (1)

[0069] (2)

[0070] The membrane potential change and pulse emission mechanism of the ARSLIF model are shown in formula (1) and formula (2).

[0071] in, V mem Initial membrane potential, V th Membrane potential threshold, τ mem Membrane potential constant, V mem_new is the updated membrane potential. In the ARSLIF model, the membrane potential changes with the input signal. Inputs When the threshold is exceeded, the neuron fires a pulse signal and the membrane potential is reset. In the ARSLIF neuron model, there are:

[0072] (3)

[0073] (4)

[0074] Where r(t) is the average emissivity, f adap (t) is the adaptive target emissivity, and the expected emissivity is defined as f .

[0075] Unlike the original LIF model, the ARSLIF neuron model V th It is not a fixed constant, but is adaptively adjusted by the error from the desired emissivity:

[0076] (5)

[0077] (6)

[0078] in, τ adapt is the time constant, α and β Adjustable hyperparameters, i represents the number of neurons, R represents the membrane resistance constant, n represents the number of neurons, and t represents the time. V th is the membrane potential threshold.

[0079] Formulas (5) and (6) show that the membrane potential threshold of the ARSLIF neuron is adaptively adjusted according to the average firing status of the neuron at the current moment, which enables the neuron model to adjust the activity level of the neuron according to different input conditions and improves the adaptability of the spiking neuron.

[0080] ARSLIF neuron model is embedded in LSTM unit:

[0081] ARSLIF neuron model replaces the traditional LSTM sigmoid The gate function maintains the stability of the pulse distribution under noise, retaining only key features, simulating the processing of speech signals by brain neurons, and effectively preventing overfitting caused by noise sensitivity and noise amplification caused by recursion. The mathematical description of the ARSLIF-BiLSTM unit is as follows:

[0082]

[0083] The ARSLIF neuron model is embedded in a Bi-LSTM neural network through dynamic threshold adjustment and membrane potential leakage mechanism, achieving efficient feature filtering and robust time series modeling in noisy environments, providing a bio-inspired interpretability solution for depression audio diagnosis.

[0084] The ARSLIF neuron model has strong anti-noise capability and good stability. The noise power of the ARSLIF neuron model is shown in formula (7):

[0085] (7)

[0086] In the formula, η(t) is the external noise. Combining formulas (5) and (6), in order to filter the pulses caused by noise, the adaptive threshold adjustment mechanism of the ARSLIF neuron model makes △Vth(t) increase with the increase of noise, and the noise power Compared with the original LIF, ARSLIF has a higher signal-to-noise ratio and better anti-noise performance.

[0087] The performance of the ARSLIF neuron model compared to other gate activation functions in noisy environments is shown in Figures 4(a)-4(f). Four gate activation functions were substituted into the RBA-FE model: sigmoid, soft threshold, standard LIF, and LIF with a desired firing rate. The performance was compared with ARSLIF. Pink noise was used to simulate low-rate ambient noise generated by traffic and electrical appliances; blue and purple noise simulated high-rate noise such as human voices. The simulated noise levels ranged from -20dB to +20dB.

[0088] Figures 4(a)-4(f) show that the ARSLIF model's root mean square error (RMSE) and mean square error (MAE) remain low at various noise levels, with minimal fluctuations, demonstrating excellent robustness. Compared to other gate activation functions, the ARSLIF model exhibits greater adaptability and stability. In particular, ARSLIF outperforms other functions in blue and purple noise environments, similar to those found in home environments. This demonstrates ARSLIF's superior noise immunity and robustness in these environments.

[0089] This example uses the Adam optimizer with a learning rate of 0.0005, and all other parameters follow the Keras default values. RMSE is selected as the loss function, and 16-bit mixed precision is used to accelerate training. The batch size (batch_size) is set to 96 to improve training efficiency. During testing, the predictions for each subject are averaged to generate the final BDI prediction. Model performance is evaluated using the RMSE and MAE metrics, comparing the predicted scores to the actual scores. The optimal parameter settings for the ARSLIF neuron model are shown in Table 1.

[0090] Table 1 Optimal parameter settings for the ARSLIF neuron model

[0091]

[0092] Comparison with other depression speech recognition methods:

[0093] The present invention designs an artificial feature extractor and an RBA-FE module to implement depression speech recognition on the AVEC 2014 dataset. As shown in Table 2, the depression speech recognition method used by the present invention outperforms other methods.

[0094] Table 2 Implementation effect of this technical solution on the AVEC 2014 dataset

[0095]

[0096] The root mean square error (RMSE) and mean square error (MAE) in Table 2 measure the difference between predicted and actual values. Smaller values ​​indicate closer agreement with the actual value, making them suitable for evaluating the performance of various algorithms. Table 2 shows that RBE-FA has a lower RMSE value and a higher MAE value. These results indicate that RBE-FA is more accurate than other models in predicting data with high depression scores (BDI), ensuring that high-risk patients are not misdiagnosed as low-risk. However, RBE-FA's MAE performance is slightly weaker, indicating that it is less effective than other regression-based models in identifying low-risk patients.

[0097] Table 3 Performance of the RBA-FE model and its variants on the AVEC 2014 dataset

[0098]

[0099] Table 3 shows that each network module in RBA-FE plays a role in effectively extracting speech features for depression diagnosis, among which the Bi-LSTM module has the greatest impact on the overall performance.

[0100] Example 2

[0101] The data set used in this embodiment is different from that in Example 1:

[0102] This example uses the MODMA audio dataset from the Second Hospital of Lanzhou University in China. The dataset consists of 52 subjects, including 23 patients with depression (16 men and 7 women) and 29 healthy controls (20 men and 9 women). The experiment includes three tasks: emotional interview questions, text reading, and image description. The audio is Mandarin, the audio sampling rate is 44.1kHz, and the experiment duration is 25 minutes. MODMA is a binary classification dataset, and all speech data is split into a training set and a data set with an 8:2 ratio. The sample data is preprocessed and feature extracted using a manual feature extractor, resulting in a [batch_size, 128, 64, 6] speech spectrum, which serves as the input for the subsequent RBA-FE.

[0103] In this embodiment, the RBA-FE module uses the Adam optimizer with a learning rate of 0.001, and the categorical cross entropy loss function is selected. Other parameters remain the same as in Example 1. Model performance is evaluated by accuracy, recall, and F1 score.

[0104] The optimal parameter settings of the ARSLIF neuron model are shown in Table 4.

[0105] Table 4 Optimal parameter settings for the ARSLIF neuron model

[0106]

[0107] Comparison with other depression speech recognition methods:

[0108] As shown in Table 5, the depression speech recognition performance of the technical solution of the present invention on the MODMA dataset is better than that of other methods.

[0109] Table 5 Implementation effect of this technical solution on MODMA dataset

[0110]

[0111] Table 5 uses precision, accuracy, recall, and F1 score for evaluation. The RBA-FE model of this solution outperforms other models in all four indicators.

[0112] Attachment Figure 5 The ROC curve of the RBA-FE model on the MODMA dataset is shown in Figure 2. The closer the curve is to the upper left corner, the better the classification performance of the model. The area under the curve (AUC) is used to measure the overall performance of the model. The closer the AUC is to 1, the better the performance. The AUC of a perfect classifier is 1. The AUC of RBA-FE is 0.89, indicating good classification performance. Figure 5 The AUC value of the RBA-FE model is 0.89, which is greater than that of other classifications, indicating that RBA-FE has the best classification performance.

[0113] By introducing the ARSLIF neuron model, this paper makes the neural network more adaptable to input audio signals and can dynamically adjust the threshold according to different signal changes, thereby effectively filtering out noise and retaining useful audio features. Furthermore, the combination of multi-head self-attention and a bidirectional LSTM network can accurately extract spatial and temporal features in audio, significantly improving the model's robustness and diagnostic accuracy. This is particularly important for self-diagnosis in home settings, enabling more accurate diagnosis of depression.

[0114] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. An audio feature extraction method based on a brain neural model, characterized in that: include: Acquire speech signal data, pre-process the speech signal data, and divide it into N speech segments; Extracting acoustic features representing the emotional state from each speech segment, the acoustic features including pitch, jitter, CQL cepstral, Mel-frequency cepstral coefficients, and first-order and second-order differential entropies of Mel-frequency cepstral coefficients, and synthesizing a multi-channel speech spectrogram after normalizing the acoustic features; The multi-channel speech spectrogram is input into the RBA-FE model, and the BDI score is output. The depression level is predicted and classified based on the BDI score. According to the difference between the predicted classification and the actual classification, the parameters of each network layer in the RBA-FE model are adjusted to obtain the optimal RBA-FE model for depression level detection. The RBA-FE model adopts a hierarchical structure, which includes a temporal convolution layer, a multi-head attention layer, and a bidirectional long short-term memory layer. Temporal convolution layer, used to extract the spatiotemporal features of multi-channel speech spectrograms; A multi-head attention layer is used to perform weighted processing on spatiotemporal features and output a global feature representation of the speech spectrum. The multi-head attention layer is added with a 75% dropout layer. A bidirectional long short-term memory layer, embedded in the ARSLIF neuron model, captures past and future contextual information in the global feature representation through independent forward and backward LSTM units, extracts the temporal dynamics of complex emotions in speech signals, and achieves a comprehensive understanding of speech sequences. The ARSLIF neuron model is obtained by introducing adaptive threshold adjustment into the standard LIF model; The ARSLIF neuron model is embedded in the bidirectional long short-term memory layer through adaptive threshold adjustment and membrane potential leakage mechanism to replace the bidirectional long short-term memory layer. Sigmoid Activation function.

2. The audio feature extraction method according to claim 1, wherein The membrane potential change mechanism of the ARSLIF neuron model is as follows: （1） （2） in, V mem Initial membrane potential, V th Membrane potential threshold, τ mem Membrane potential constant, V mem_new is the updated membrane potential; In the ARSLIF neuron model, the membrane potential changes with the input signal. Inputs When the threshold is exceeded, the neuron fires a pulse signal and the membrane potential is reset, as follows: （3） （4） Where r(t) is the average emissivity, f adap (t) is the adaptive target emissivity, and the expected emissivity is defined as f , the active firing rate of the ARSLIF neuron model f active and the resting firing rate of the ARSLIF neuron model f rest , i represents the number of neurons, R represents the membrane resistance constant, n is the number of neurons, t is the time, V th is the membrane potential threshold.

3. The audio feature extraction method according to claim 1, wherein The adaptive threshold adjustment is to adaptively adjust the membrane potential threshold by calculating the error with the expected firing rate. V th , specifically: ARSLIF neuron model V th Rather than being a fixed constant, it is adaptively adjusted by the error from the desired emissivity: （5） （6） in, τ adapt is the time constant, α and β Adjustable hyperparameters, r e is the rate error, V th_new is the new threshold after update.

4. The audio feature extraction method according to claim 1, wherein The temporal convolution layer includes multiple cascaded convolution layers and pooling layers.

5. The audio feature extraction method according to claim 4, wherein: The specific process of the temporal convolution layer is as follows: The acoustic features are convolved with a 7×7 convolution kernel to extract the comprehensive features of the speech features by expanding the receptive field. After batch normalization, ReLU activation and average pooling, they are input into the residual network with a 3×3 convolution kernel to further extract the detailed features of the speech signal.

6. The audio feature extraction method according to claim 1, wherein: The mean absolute error and mean squared error are used to evaluate the difference between the predicted classification and the actual classification.

7. The audio feature extraction method according to claim 1, wherein: The multi-head attention layer includes eight attention heads arranged in parallel to learn different acoustic features.

Images