Cross-domain depression detection system based on time-frequency calibration transfer learning

Abstract

The invention discloses a cross-domain depression detection system based on time-frequency calibration transfer learning. According to the method, firstly, a frequency delay neural network and a time delay neural network are used for respectively extracting frequency domain and time domain features of voice signals, and local and global time-frequency embedding representations are fused through a multi-layer information aggregation module, so that a comprehensive vector capable of representing depression features is obtained. And then, designing a distribution calibration transfer learning module, and through construction of a high-confidence sample pair, reducing the distribution distance between samples of the same kind and enlarging the distribution difference between samples of different kinds, and maintaining the consistency and discrimination of depression features in a domain transfer process. According to the method, the depression detection precision is remarkably improved in cross-corpus and cross-speaker scenes, the problem that a traditional method is insufficient in generalization ability under the situation of domain mismatching is solved, and the method has good robustness and clinical application prospects.

Description

Technical Field

[0001] This invention relates to a cross-domain depression detection method and system based on time-frequency calibration transfer learning, belonging to the field of speech depression detection technology. Background Technology

[0002] Depression is a common mental illness affecting approximately 280 million people worldwide, and in severe cases, it can lead to suicide. Although effective treatments exist for mild, moderate, and severe depression, in low- and middle-income countries, over 75% of patients do not receive treatment due to a lack of professional mental health professionals and other factors. Therefore, monitoring individual mental health and detecting depressive symptoms as early as possible is crucial. Since the process of speech production can be influenced by cognitive and physiological factors caused by depression, acoustic features are an effective diagnostic tool for depression. Furthermore, communicating with patients non-contactly using only a microphone improves accessibility and protects patient privacy. Therefore, speech-based diagnostic technologies for depression are gaining popularity and can be combined with current diagnostic techniques to improve diagnostic accuracy.

[0003] In recent years, significant progress has been made in speech depression detection based on deep learning. However, training deep neural network models typically relies on large and diverse data samples. Due to privacy restrictions on depressive corpora and the low proportion of depressive samples in the overall population, the available training datasets for depression are small in size and severely imbalanced in class distribution, thus affecting the robustness and generalization ability of the models. Furthermore, differences in data distribution across different corpora, recording conditions, or speakers can cause a model trained in one domain to perform significantly worse in another. To alleviate these problems, existing methods typically employ data-level resampling, cost-sensitive loss weighting, or domain-alignment-based transfer learning, such as gradient inversion and adversarial domain adaptation. However, these methods have limitations in practice: resampling may introduce noise or lead to overfitting; simple global distribution alignment can easily smooth out inter-class discriminative power; and single time-domain or frequency-domain modeling may miss important complementary information. Therefore, this paper proposes to improve the generalization ability and discriminative power of depression detection models under domain transformation conditions by jointly performing hierarchical feature extraction in the frequency / time domain, multi-layer information aggregation, and distribution calibration transfer learning.

[0004] This system introduces a time-frequency calibration transfer learning framework into the speech depression detection task. Through a joint time-domain-frequency domain hierarchical modeling and distributed calibration transfer mechanism, it specifically addresses the problems of data scarcity, class imbalance, and domain discrepancies. A frequency-delayed neural network (FDNN) is designed on the frequency axis to supplement the shortcomings of traditional time-domain models; a time-delayed neural network (TDNN) is used on the time axis to extract temporal features; a multi-layer information aggregation module (MIA) fuses time-frequency embeddings from different time layers to preserve the complementarity of local and global information; and a distributed calibration transfer learning module (DCTL) is introduced in the transfer learning stage. Based on high-confidence positive / negative sample pairs selected from the source domain, a triplet calibration constraint is constructed and jointly optimized with a gradient-reversed domain classification loss. This enhances both source / target domain alignment and class-like aggregation and dissimilar-like separation, thereby avoiding damage to the depressive / non-depressive discrimination information during the alignment process. Summary of the Invention

[0005] This invention provides a cross-domain depression detection system based on time-frequency calibration transfer learning. In practical applications, speech depression detection not only faces the problems of scarce depression samples and class imbalance, but also the challenge of inconsistent source and target domain distributions in scenarios such as cross-corpus and cross-speaker scenarios. These problems lead to a significant decline in model performance in new domains, reducing detection accuracy and robustness. To address these issues, this invention proposes a time-frequency joint transfer learning framework: frequency-delayed neural network (FDNN) and time-delayed neural network (TDNN) extract frequency and time domain features respectively, then the multi-layer information aggregation module (MIA) fuses local and global embedding representations, and finally, the distribution calibration transfer learning module (DCTL) is combined to maintain class discriminativeness while aligning the feature distributions of different domains, thereby effectively improving the performance of cross-domain depression detection.

[0006] The user process of the cross-domain depression detection system based on time-frequency calibration transfer learning provided by this invention includes the following steps: First, the input speech signal is preprocessed to extract logarithmic Mel filter bank features, forming a two-dimensional feature matrix. Then, it is input into a Frequency Delay Neural Network (FDNN) and a Time Delay Neural Network (TDNN) respectively, extracting hierarchical feature representations on the frequency and time axes, and obtaining embedding vectors in the frequency and time domains through statistical pooling. Next, the embeddings of each layer are concatenated and refined in the Multi-Layer Information Aggregation (MIA) module to obtain a comprehensive time-frequency embedding representation. Subsequently, the embedding space is calibrated in the Distributed Calibration Transfer Learning (DCTL) module using triplet loss and gradient inversion strategies to reduce the distance between similar samples and increase the distance between dissimilar samples, achieving cross-domain alignment. During training, the calibrated embeddings are input into a classifier, outputting the predicted probability of depression or non-depression, and the system parameters are optimized by combining the depression classification loss and transfer calibration loss. After training, acoustic features are extracted from the data of the subjects to be tested, and two-dimensional acoustic feature samples are constructed, which are then sent to the system for testing to obtain the final depression category. The technical solution adopted in this invention can be further refined. The aforementioned process for constructing the feature matrix X is specifically as follows: First, the input raw speech signal is pre-emphasized and then framed with a frame length of 25ms and a frame shift of 10ms. Each frame is multiplied by a Hamming window to form a windowed signal. A Fast Fourier Transform (FFT) is then performed on the signal to obtain the power spectrum of each frame. This power spectrum represents the energy distribution of the signal at different frequency points. Let the number of FFT points be N, then the frequency index is k = 0, 1, 2, ..., N / 2. The formula for calculating the power spectrum of the m-th frame is: Among them, P m (k) represents the power spectrum of the m-th frame at the k-th frequency point, and FFT{·} represents the Fast Fourier Transform operation, which transforms the time-domain signal to the frequency domain. m (n) represents the windowed speech signal of the m-th frame, where n represents the index of the sampling point within the frame. The power spectrum is processed through F Mel filter banks to calculate the energy, and then the logarithm is taken to obtain the logarithmic Mel filter bank characteristic X. m (f): Among them, H f (k) represents the response weight of the f-th Mel filter at frequency point k, where f = 1, 2, 3...F; The feature matrix X∈R is obtained by arranging the features of all frames in chronological order. T×F Where T is the number of frames and F is the number of Mel frequency bands. The feature extraction process of the frequency delay neural network (FDNN) mentioned above is specifically as follows: In the i-th layer, along the frequency axis, the feature matrix X... s Perform convolution operation on each time frame: in, This represents the frequency domain convolution kernel of the i-th layer, whose output channel number determines the channel dimension of the feature map. The symbol * indicates a convolution operation. The resulting feature map... Preserving the time dimension, the product is then processed sequentially using the non-linear activation function ReLU, batch normalization (BN) (·), and dropout (D) (·) to obtain... To obtain a fixed-length representation from the features of these time distributions, statistical pooling is performed on the time dimension, and the statistics are concatenated to form a frequency domain feature embedding: in, and Let represent the mean and standard deviation of the i-th layer frequency domain feature map in the time dimension, respectively. The feature extraction process of the Time Delay Neural Network (TDNN) mentioned above is specifically as follows: In the i-th layer, a convolution operation is performed on the feature matrix X along the time axis: in, Let represent the temporal convolution kernel of the i-th layer, after processing with the ReLU activation function, batch normalization, and Dropout operation, to obtain... right Statistical pooling is performed over the time dimension to obtain the mean. with standard deviation And concatenate them into a time-domain embedding: As the temporal embedding vector of the i-th layer. The aforementioned multi-layer information aggregation module (MIA) specifically involves concatenating time-domain embeddings and frequency-domain embeddings at the same level. H i =concat(T i ,TF i ); The obtained H i Input depth is D i The fully connected network yields refined feature vectors: Refine the feature vectors E at all levels i By cascading, we obtain the time-frequency synthesized embedded representation e = concat(E i ). The aforementioned Distribution Calibration Transfer Learning (DCTL) module specifically involves: selecting high-confidence depressed and non-depressed samples from the source domain samples to construct positive and negative sample pairs; and using a triplet loss function for calibration, defined as: L cal =max(D(E) s ,e position )-D(E s ,e negative )+m,0) Where D(·,·) is the Euclidean distance function, e position To embed E s high-confidence sample embeddings belonging to the same category, e neative To embed E s The high-confidence sample embeddings belonging to the outlier class are used, where m is the margin parameter. Simultaneously, the domain classification loss is calculated, including the source domain classification loss and the target domain classification loss: Among them, L s Represents the source domain classification loss and L t p represents the target domain classification loss. sou p target Let y represent the probabilities that the domain classifier predicts a sample from the source domain and the target domain as belonging to the source domain, respectively. sour and y target Let the true labels in the source and target domains be represented respectively; summing them yields the gradient reversal loss: L GradReverse =L s +L t Then, the triplet calibration loss is combined with the gradient reversal loss to obtain the calibrated gradient reversal loss: L CGR =L GradReverse +L cal Simultaneously calculate the depression classification loss: L depression = -[ylogp+(1-y)log(1-p)], where y represents the true depression label, and p represents the probability that the classifier predicts the sample belongs to the depression category. The final optimization objective is the weighted sum of the depression classification loss and the calibration gradient reversal loss. L total =βL depression +αL CGR Where α and β represent weighting coefficients, respectively. The classifier construction process mentioned above is as follows: the calibrated embedding vector e is input into a fully connected layer to obtain a linear transformation output z, which is then mapped using a sigmoid function to obtain the depression probability. p = σ(z) Where z represents the output value of the linear transformation, σ represents the Sigmoid activation function, and p is the predicted probability that the sample belongs to the depression category. The above-mentioned update process for training the voice depression detection model is as follows: the model is updated through backpropagation, and the parameters are saved after training is completed.

[0007] The beneficial effects of this invention are as follows: Compared with existing speech depression detection methods, this invention has significant advantages in the following aspects: First, by combining Frequency Delay Neural Network (FDNN) and Time Delay Neural Network (TDNN), this invention can simultaneously capture key information of speech signals in both the frequency and time domains, overcoming the deficiency of single feature extraction methods in expressing depression-related features. Second, this invention introduces hierarchical embedding and deep fully connected refinement mechanisms in the Multilayer Information Aggregation (MIA) module, enhancing the fusion ability of local and global features and effectively improving the discriminative power of depression features. Furthermore, the Distribution Calibration Transfer Learning (DCTL) module proposed in this invention during the transfer stage, through joint optimization of triplet loss and gradient inversion strategy, can reduce the distribution difference between the source and target domains, achieving domain alignment while maintaining depression-related information, thereby significantly improving the robustness of cross-domain detection. Finally, this invention uses the weighted sum of depression classification loss and calibration transfer loss as the optimization objective, enabling the model to achieve a balance between depression detection accuracy and cross-domain generalization ability. Attached Figure Description

[0008] Figure 1 This is a flowchart of a cross-domain depression detection system based on time-frequency calibration transfer learning.

[0009] Figure 2 This is a network model diagram of the Frequency Delayed Neural Network (FDNN) module.

[0010] Figure 3 This is the network model diagram of the Distributed Calibration Transfer Learning (DCTL) module. Detailed Implementation

[0011] To more clearly describe the content of this invention, further explanation is provided below with reference to examples and accompanying drawings. The embodiments described below are not intended to limit the scope of this invention. A cross-domain depression detection system based on time-frequency calibration transfer learning according to this invention includes the following steps:

[0012] Step 1: Preprocess and calculate features of the raw speech signal to construct a feature matrix X∈R for subsequent network input. T×F The specific steps are as follows:

[0013] 1.1 The original speech is first pre-emphasized, then framed with a frame length of 25ms and a frame shift of 10ms. Each frame is multiplied by a Hamming window to form a windowed signal. A Fast Fourier Transform (FFT) is performed on the signal to obtain the power spectrum of each frame. This power spectrum represents the energy distribution of the signal at different frequency points. Let the number of FFT points be N, then the frequency index is k = 0, 1, 2, ..., N / 2. The formula for calculating the power spectrum of the m-th frame is: Among them, P m (k) represents the power spectrum of the m-th frame at the k-th frequency point, and FFT{·} represents the Fast Fourier Transform operation, which transforms the time-domain signal to the frequency domain. m (n) represents the windowed speech signal of the m-th frame, where n represents the index of the sampling point within the frame. The power spectrum is processed through F Mel filter banks to calculate the energy, and then the logarithm is taken to obtain the logarithmic Mel filter bank characteristic X. m (f): Among them, H f (k) represents the response weight of the f-th Mel filter at frequency point k, where f = 1, 2, 3...F; the feature matrix X∈R is obtained by arranging the features of all frames in chronological order. T×F Where T is the number of frames and F is the number of Mel frequency bands. The resulting feature matrix X s This serves as the common input for subsequent FDNN and TDNN.

[0014] Step 2: Construct a Frequency Delay Neural Network (FDNN) and extract frequency domain features. The specific steps are as follows:

[0015] 2.1 Frequency Delay Neural Network (FDNN) is mainly used to extract frequency domain features of speech along the frequency axis. It consists of multiple one-dimensional convolutional blocks. Each convolutional block includes, in sequence: a convolutional layer, a ReLU activation function, a batch normalization layer, and a Dropout layer. The first convolutional layer has a kernel size of 5, a dilation factor of 1, and 512 output channels. It then passes through a ReLU activation function and a batch normalization layer with a dropout of 0.1. The second convolutional layer has a kernel size of 3, a dilation factor of 2, and 512 output channels. It then passes through a ReLU activation function and a batch normalization layer with a dropout of 0.1. The third convolutional layer has a kernel size of 3, a dilation factor of 3, and 512 output channels. It then passes through a ReLU activation function and a batch normalization layer with a dropout of 0.1. The fourth convolutional layer has a kernel size of 1, a dilation factor of 1, and 512 output channels. It then passes through a ReLU activation function and a batch normalization layer with a dropout of 0.1. The fifth convolutional layer has a kernel size of 1, a dilation factor of 3, and 512 output channels. It then passes through a ReLU activation function and a batch normalization layer with a dropout of 0.1.

[0016] 2.2 After the network is constructed, the FDNN is responsible for performing hierarchical convolutions along the frequency axis to extract frequency domain features. The specific steps are as follows:

[0017] 2.3 Input: Feature matrix X∈R T×F Where T is the number of frames and F is the number of Mel frequency bands. For each time frame, the FDNN performs convolution operations in the frequency direction to capture local patterns in the frequency domain.

[0018] 2.4 The frequency domain convolution of the i-th layer is defined as: in, This represents the frequency domain convolution kernel of the i-th layer, whose output channel number determines the channel dimension of the feature map. The symbol * indicates a convolution operation. The resulting feature map... Preserving the time dimension, the product is then processed sequentially using the non-linear activation function ReLU, batch normalization (BN) (·), and dropout (D) (·) to obtain... To obtain a fixed-length representation from the features of these time distributions, statistical pooling is performed on the time dimension, and the statistics are concatenated to form a frequency domain feature embedding: in, and Let represent the mean and standard deviation of the i-th layer frequency domain feature map in the time dimension, respectively.

[0019] Step 3: Construct a Temporal Delayed Neural Network (TDNN) and extract temporal features. The specific steps are as follows:

[0020] 3.1 Temporal Delay Neural Network (TDNN) is mainly used to extract temporal features of speech along the time axis. It consists of multiple one-dimensional convolutional blocks. Each convolutional block includes, in sequence: a convolutional layer, a ReLU activation function, a batch normalization layer, and a Dropout layer. The first convolutional layer has a kernel size of 5, a dilation factor of 1, and 512 output channels. It then passes through a ReLU activation function and a batch normalization layer with a dropout of 0.1. The second convolutional layer has a kernel size of 3, a dilation factor of 2, and 512 output channels. It then passes through a ReLU activation function and a batch normalization layer with a dropout of 0.1. The third convolutional layer has a kernel size of 3, a dilation factor of 3, and 512 output channels. It then passes through a ReLU activation function and a batch normalization layer with a dropout of 0.1. The fourth convolutional layer has a kernel size of 1, a dilation factor of 1, and 512 output channels. It then passes through a ReLU activation function and a batch normalization layer with a dropout of 0.1. The fifth convolutional layer has a kernel size of 1, a dilation factor of 3, and 512 output channels. It then passes through a ReLU activation function and a batch normalization layer with a dropout of 0.1.

[0021] 3.2 After the network is built, the TDNN will be responsible for performing hierarchical convolutions along the frequency axis to extract temporal features. The specific steps are as follows:

[0022] 3.3 Perform convolution operation on the feature matrix X along the time axis in the i-th layer: in, Let represent the temporal convolution kernel of the i-th layer, after processing with the ReLU activation function, batch normalization, and Dropout operation, to obtain... right Statistical pooling is performed over the time dimension to obtain the mean. with standard deviation And concatenate them into a time-domain embedding: TDNN provides multiple layers of time-series information for MIA aggregation.

[0023] Step 4: Construction and initialization of the Multi-Level Information Aggregation Module (MIA). The MIA is mainly used to fuse frequency domain embeddings and time domain embeddings at different levels to obtain a more discriminative comprehensive representation. The specific steps are as follows:

[0024] 4.1 First, the frequency domain embedding vector TF of the i-th layer is... i With the temporal embedding vector T i By concatenating along the feature dimensions, we obtain the fused representation H. i =concat(T i ,TF i ), then H i Input to depth D i A fully connected FC network is constructed, and refined feature vectors are obtained by performing nonlinear transformation and regularization using the ReLU activation function and Dropout operation. Finally, the refined feature vectors {E1, E2, ..., E...} obtained from each level are... L Cascaded operations are performed to obtain the synthesized time-frequency embedding representation E. s =concat(E i This serves as input for subsequent transfer learning and classification.

[0025] Step 5: Construction and initialization of the Distribution Calibration Transfer Learning (DCTL) module. DCTL is mainly used to align feature distributions between the source and target domains while maintaining the discriminative power between depressed and non-depressed categories. The specific steps are as follows:

[0026] 5.1 First, select high-confidence depressed and non-depressed samples from the source domain samples to construct positive and negative sample pairs; calibrate using the triplet loss function, defined as: L cal =max(D(E) s ,e position )-D(E s ,e negative )+m,0) Where D(·,·) is the Euclidean distance function, e position To embed E s high-confidence sample embeddings belonging to the same category, e neati To embed E s The high-confidence sample embeddings belonging to the outlier class are used, where m is the margin parameter. Simultaneously, the domain classification loss is calculated, including the source domain classification loss and the target domain classification loss: Among them, L s Represents the source domain classification loss and L t p represents the target domain classification loss. sourc p target Let y represent the probabilities that the domain classifier predicts a sample from the source domain and the target domain as belonging to the source domain, respectively. sou and y target Let each represent the true label of the domain; add them together to obtain the gradient reversal loss: L GradReverse=L s +L t Then, the triplet calibration loss is combined with the gradient reversal loss to obtain the calibrated gradient reversal loss: L CGR =L GradReverse +L cal Simultaneously calculate the depression classification loss: L depression = -[ylogp+(1-y)log(1-p)], where y represents the true depression label, and p represents the probability that the classifier predicts the sample belongs to the depression category. The final optimization objective is the weighted sum of the depression classification loss and the calibration gradient reversal loss. L total =βL depression +αL CGR Where α and β represent weighting coefficients, respectively.

[0027] Step 6: Classifier construction, the specific steps are as follows: Input the calibrated embedding vector e into the fully connected layer to obtain the linear transformation output z, and then obtain the depression probability by mapping through the Sigmoid function: p = σ(z) Where z represents the output value of the linear transformation, σ represents the Sigmoid activation function, and p is the predicted probability that the sample belongs to the depression category.

[0028] Step 7: The process of updating the voice depression detection model is as follows: update the model through backpropagation and save the parameters after training is completed.

[0029] Step 8: After the depression detection model is trained, acoustic features are extracted from the data of the subjects to be tested and two-dimensional acoustic feature samples are constructed. Then, these samples are sent into the system for testing to obtain the final depression category.

[0030] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention. Experimental Design

[0031] Experimental Dataset and Partitioning: The datasets used in the experiment are the Distress Analysis Interview Corpus, Wizard of Oz (DAIC-WoZ) and the Chinese Depression Corpus (CMDC). DAIC-WoZ is the official dataset designated by the AVEC2016 Depression Detection Challenge. This dataset is a publicly available English depression dataset containing audio and text transcriptions of 189 participants. Each audio file ranges in length from 7 to 33 minutes, with a fixed sampling rate of 16000Hz. Participants were labeled using PHQ-8 scores based on their questionnaire responses; participants with scores greater than or equal to 10 were considered to have depression. The DAIC-WoZ dataset includes a training set of 107 participants (30 depressed, 77 non-depressed) and a development set of 35 participants (12 depressed, 25 non-depressed). The CMDC dataset contains 87 Chinese participants, of whom 26 exhibited depressive tendencies and 61 were non-depressive. The corpus comprises 924 Chinese audio recordings and corresponding text transcriptions, with a fixed sampling rate of 16000Hz. Depressive tendencies were determined using the HAMD score. For CMDC, the same processing method as the original authors was maintained, employing five-fold cross-validation to train and evaluate the model to improve robustness and statistical reliability. Accuracy, precision, recall, and the harmonic mean of precision and recall (F1 score) were used as evaluation metrics for the model. Speakers with a PHQ-8 score of 15 or higher were considered high-confidence depressive data, while those with a score below 5 were considered high-confidence non-depressive data. For the HAMD-17 scale, scores of 18 or higher and 7 or lower were used to define high-confidence depressive and high-confidence non-depressive samples, respectively. Experimental results

[0032] Experimental Results under Different Voice Depression Detection Networks: The cross-domain depression detection method based on time-frequency calibration transfer learning proposed in this invention was compared with several existing voice depression detection methods. The results are shown in Tables 1 and 2. Compared with existing state-of-the-art voice depression detection methods such as Random Forest, FVTC-CNN, MT-DLR-CombAtt, MFCC-RNN, LSTM, STFN, 1DCNN+Transformer, GRU-BiLSTM, BiLSTM, CubeMLP, PMR, and IIFDD, the proposed method shows varying degrees of improvement in precision, recall, and F1 score. On the DAIC-WoZ dataset, the proposed method achieves the best results in terms of accuracy and precision: ACC = 0.8175, Precision = 0.7136. While comparable to the current best method STFN in terms of F1 score, it better balances precision and recall, demonstrating higher robustness. On the CMDC dataset, the method of this invention outperforms methods including GRU-BiLSTM, BiLSTM, PMR, and CubeMLP in F1 = 0.9660, Precision = 0.9648, and Recall = 0.9675. It also improves the F1 score by 5% compared to the current best method, IIFDD, and by 6.2% compared to Wav2vec. The experimental results fully demonstrate the effectiveness and advancement of this method in cross-domain speech depression detection. Table 1. Indicator values ​​of the method of the present invention and various comparison algorithms on the DAIC-WoZ dataset. Table 2. Indicator values ​​of the method of the present invention and various comparative algorithms on the CMDC dataset.

Claims

1. A cross-domain depression detection system based on time-frequency calibration transfer learning, characterized in that, Includes the following steps: S1 preprocesses the original speech signal and extracts the log-Mel filter bank features to form a feature matrix X, denoted as X∈R. T ×F Where T is the number of frames and F is the number of Mel frequency bands; S2 feeds the feature matrix X into a frequency delay neural network (FDNN), performs multi-layer convolution operations along the frequency axis to extract frequency domain features, and performs statistical pooling on each layer of features in the time dimension to obtain the frequency domain embedding vector TF of the i-th layer. i ; S3 feeds the feature matrix X into a time-delayed neural network (TDNN), performs multi-layer convolution operations along the time axis to extract temporal features, and performs statistical pooling on each layer of features in the time dimension to obtain the temporal layer embedding vector T of the i-th layer. i ; S4 embeds the frequency domain level into TF for each selected level i in the multi-layer information aggregation module MIA. i With temporal hierarchical embedding T i spliced ​​into H i and for H i The refined feature vector E is obtained by further refining the feature vector through either a deep or shallow fully connected network. i ; S5 refines the feature vector E of the i-th layer. i Cascading yields the time-frequency synthesized embedded representation E s ; S6 performs transfer learning on source domain samples and target domain samples. In the transfer phase, a distributed calibration transfer learning module (DCTL) is introduced. Sample pairs are constructed by selecting high-confidence depressed samples and high-confidence non-depressed samples from the source domain data. The embedding space is calibrated using triple constraints to obtain the calibrated time-frequency integrated embedding e. S7 embeds the calibrated time-frequency synthesis into the e-input classifier, and the classifier outputs the predicted probability p of whether the sample is depressed or not; S8 uses the weighted sum of depression classification loss and calibration transfer loss as the training objective, performs backpropagation and optimization on the model parameters, and saves the model parameters after training. During the testing phase, S9 loads the trained model and processes the test subject's speech through S1-S5 to obtain the time-frequency integrated embedding E of the test speech. s The input classifier obtains the depression probability of the test speech. If the depression probability is higher than the non-depression probability, the user of the test speech is considered to have a depressive tendency. If the test subject provides multiple speech segments, the depression probabilities of the multiple speech segments are voted on or thresholded to output the final depression detection result.

2. The cross-domain depression detection system based on time-frequency calibration transfer learning according to claim 1, characterized in that, The construction process of the feature matrix X in step S1 is as follows: After pre-emphasizing the input raw speech signal, it is divided into frames with a frame length of 25ms and a frame shift of 10ms. Each frame is multiplied by a Hamming window to form a windowed signal. Then, a fast Fourier transform is performed to obtain the power spectrum of each frame. This power spectrum represents the energy distribution of the signal at different frequency points. Let the number of points of the fast Fourier transform be N, then the frequency point index is k = 0, 1, 2, ..., N / 2. The formula for calculating the power spectrum of the m-th frame is: Among them, P m (k) represents the power spectrum of the m-th frame at the k-th frequency point, and FFT{·} represents the Fast Fourier Transform operation, which transforms the time-domain signal to the frequency domain. m (n) represents the windowed speech signal of the m-th frame, where n represents the intra-frame sampling point index; the power spectrum of the m-th frame is processed through F Mel filter banks to calculate the energy, and then the logarithm is taken to obtain the logarithmic Mel filter bank characteristic X. m (f): Among them, H f (k) represents the response weight of the f-th Mel filter at frequency point k, where f = 1, 2, 3...F; X of all frames m (f) The features are arranged in chronological order to obtain the feature matrix X∈R. T×F Where T is the number of frames and F is the number of Mel frequency bands.

3. The cross-domain depression detection system based on time-frequency calibration transfer learning according to claim 1, characterized in that, The calculation of the Frequency Delayed Neural Network (FDNN) in step S2 is specifically as follows: In the i-th layer, a convolution operation is performed on each time frame of the feature matrix X along the frequency axis: in, This represents the frequency domain convolution kernel of the i-th layer, whose output channel number determines the channel dimension of the feature map. The symbol * represents the convolution operation, which produces the feature map. Preserving the time dimension, the process then sequentially passes through a non-linear activation function, batch normalization, and Dropout to obtain... To obtain a fixed-length representation from the features of these time distributions, statistical pooling is performed on the time dimension, and the statistics are concatenated to form a frequency domain feature embedding: in, and Let represent the mean and standard deviation of the i-th layer frequency domain feature map in the time dimension, respectively.

4. The cross-domain depression detection system based on time-frequency calibration transfer learning according to claim 1, characterized in that, The construction process of the Time Delay Neural Network (TDNN) described in step S3 is as follows: In the i-th layer, a convolution operation is performed on the feature matrix X along the time axis: in, Let represent the temporal convolution kernel of the i-th layer, which, after processing with a non-linear activation function, batch normalization, and Dropout operation, yields... right Statistical pooling is performed over the time dimension to obtain the mean. with standard deviation And concatenate them into a time-domain embedding:

5. A cross-domain depression detection system based on time-frequency calibration transfer learning according to claim 1, characterized in that, The construction process of the multi-layer information aggregation module MIA in step S4 is specifically as follows: the time-domain embedding and the frequency-domain embedding are concatenated at the i-th layer to obtain H. i =concat(T i ,TF i ); for H i Input depth is D i A fully connected network is used to obtain refined feature vectors. Refine the feature vectors E at all levels i By cascading, the time-frequency integrated embedded representation E is obtained. s =concat(E i ).

6. The cross-domain depression detection system based on time-frequency calibration transfer learning according to claim 1, characterized in that, The construction process of the Distribution Calibration Transfer Learning (DCTL) module described in step S5 is as follows: High-confidence depressed and non-depressed samples are selected from the source domain samples to construct positive and negative sample pairs; calibration is performed using a triplet loss function, defined as: IT cal =max(D)E s ,it position )-OF s ,it negative )+m,0) Where D(·,·) is the Euclidean distance function, e position For time-frequency integrated embedding representation E s high-confidence sample embeddings belonging to the same category, e neative For time-frequency integrated embedding representation E s The high-confidence sample embeddings belonging to the outlier class are used, where m is the margin parameter. Simultaneously, the domain classification loss is calculated, including both the source domain classification loss and the target domain classification loss. Among them, L s Represents the source domain classification loss and L t p represents the target domain classification loss. sourc p target Let y represent the probabilities that the domain classifier predicts a sample from the source domain and the target domain as belonging to the source domain, respectively. sourc and y target Let the true labels in the source and target domains be represented respectively; summing them yields the gradient reversal loss: L GradReverse =L s +L t Then, the triplet calibration loss is combined with the gradient reversal loss to obtain the calibrated gradient reversal loss: L CGR =L GradReverse +L cal Simultaneously calculate the depression classification loss: L depression = -[ylogp+(1-y)log(1-p)], where y represents the true depression label, p represents the probability that the sample belongs to the depression category as predicted by the classifier, and the final optimization objective is the weighted sum of the depression classification loss and the calibration gradient reversal loss: L total =βL depression +αL CGR Where α and β represent weighting coefficients, respectively.

7. A cross-domain depression detection system based on time-frequency calibration transfer learning according to claim 1, characterized in that, The classifier construction process described in step S7 is as follows: the accurate time-frequency integrated embedding e is input into the fully connected layer to obtain the linear transformation output z, which is then mapped to the depression probability p through the Sigmoid function.

8. A cross-domain depression detection system based on time-frequency calibration transfer learning according to claim 1, characterized in that, The depths of the frequency delay neural network, the time delay neural network, and each fully connected network in the MIA are all adjustable parameters, and their specific values ​​can be set according to application requirements.

9. A cross-domain depression detection system based on time-frequency calibration transfer learning according to claim 1, characterized in that, The distinction between high-confidence depressed samples and high-confidence non-depressed samples is based on the scoring results of clinical scales, and is determined by setting a scoring threshold to construct sample pairs accordingly.

Images