Speech-based depression detection model construction method and depression detection system

Abstract

This invention discloses a method for constructing a speech-based depression detection model and a depression detection system. By constructing a depression detection model and using speech data for speech depression detection, this method enables end-to-end learning of depressive features present in speech, while also enabling the quantification and visualization of depression biomarkers. Furthermore, to address the weak interpretability of the model due to limited clinical data, this invention employs a novel speech self-supervised learning strategy to improve feature interpretability and model generalization.

Description

Technical Field

[0001] The present invention relates to the intersection of psychiatry, artificial intelligence and machine learning, and in particular to a method for constructing a speech-based depression detection model and a depression detection system. Background Art

[0002] Depression is a common mental health problem and a significant contributor to the global burden of disease. Depression typically begins in adolescence, may persist or relapse in adulthood, and often becomes a lifelong, chronic mental disorder. Currently, the detection and screening of depression primarily rely on subjective methods such as questionnaires and clinical interviews. These methods, for example, are based on instruments such as the Patient Health Questionnaire-9 (PHQ-9), the Hamilton Depression Rating Scale 17-item (HAMD-17), and the Depression Anxiety Stress Scale-21 (DASS-21). However, these methods are easily affected by interviewer experience, questionnaire quality, and participant willingness. Therefore, there is a need to develop objective methods for detecting depression.

[0003] Speech, as a readily available biometric data, offers an effective perspective for depression detection, as various emotional states can be expressed through distinct vocalization patterns. Previous studies have shown that the speech of individuals with depression exhibits lower pitch, slower pitch changes, slurred speech, or long pauses. Therefore, designing a speech-based depression detection (SDD) method to identify acoustic biomarkers associated with depression is of great practical significance for improving detection performance.

[0004] Recently, there has been significant development of deep learning (DL) methods for SDD tasks, which can be divided into two main categories: developing models based on manually extracted spectral features, and building models that can directly process raw signals.

[0005] 1) The first category of methods uses spectral features such as power spectrograms, Mel filter banks (MFbanks) and MFCCs as input, and processes these features through convolutional neural networks (CNNs) and long short-term memory networks (LSTMs) combined with attention mechanisms to mine and identify key local speech features.

[0006] 2) The second category of methods avoids relying on handcrafted spectral features and instead learns features directly from the raw signal. For example, multiple convolutional layers are used to reduce the dimensionality of the speech signal, generating an input of similar size to the spectral features. However, due to the high dimensionality of the input signal, such models typically require larger convolution kernels and more complex network structures, resulting in an excessive number of model parameters.

[0007] Furthermore, while SDD methods based on the aforementioned deep learning approaches have made some progress, a major limitation of these models is their lack of interpretability, meaning they cannot identify meaningful depressive speech patterns or acoustic indicators. In reality, in actual clinical practice, the discovery of objective biomarkers can provide more clinical support for depression detection than simply improving prediction accuracy by a small margin. Summary of the Invention

[0008] The present invention aims to solve one of the technical problems existing in the related art at least to a certain extent.

[0009] An object of the present invention is to provide a method for constructing a speech-based depression detection model, and to construct an interpretable depression screening network framework (Learning Interpretable Depression Representations in Speech, DisNet) for depression detection using speech. The DisNet framework can learn depressive features present in speech end-to-end. At the same time, by using a novel speech self-supervised learning strategy (SLRD), the interpretability of the features and the generalization performance of the model are improved, thereby solving the problem of weak model interpretability caused by insufficient clinical data.

[0010] Another object of the present invention is to provide a depression detection system using the above-mentioned construction method to perform speech-based depression detection and achieve quantification and visualization of depression biomarkers.

[0011] In order to achieve the above-mentioned objectives, the present invention provides a method for constructing a speech-based depression detection model, comprising the following steps:

[0012] S100, collecting voice data and adding labels to indicate whether the voice is depressive; preprocessing the collected voice data to construct a voice dataset;

[0013] S200, constructing a depression screening network framework, wherein the depression screening network framework includes at least a learnable frequency domain filter bank module for extracting spectral features related to depression from speech signals, and a hierarchical speech feature extraction module for extracting depression-related representations and locating key pronunciation areas;

[0014] The speech data is input into the depression screening network framework, and the learnable frequency domain filter bank module selects spectral features related to depression from the speech data by adaptively adjusting the frequency response of the filter;

[0015] Input the spectral features extracted by the learnable frequency domain filter bank module into the hierarchical speech feature extraction module;

[0016] The hierarchical speech feature extraction module processes the input spectral features in blocks and performs compression and sparsification operations through forward propagation, gradually discarding features irrelevant to the depression detection task and retaining key depression-related features;

[0017] S300, conduct two-stage training on the constructed depression screening network framework;

[0018] The first stage is the pre-training stage. It uses an unsupervised learning strategy to randomly sample speech samples from the speech dataset and extract non-overlapping segments as input data for the depression screening network framework. The sampled segments are input into a learnable frequency domain filter bank module and a hierarchical speech feature extraction module to extract emotion representation and speaker representation. The extracted representations are then interacted and reconstructed. The model parameters of the depression screening network framework are updated according to the designed loss function, and the weights of the depression screening network framework are saved.

[0019] The second stage is the fine-tuning stage. The network weights in the pre-training stage are used to initialize the depression screening network framework. A self-supervised learning strategy is adopted. On a labeled speech dataset, the model parameters of the depression screening network framework are fine-tuned using known labels to obtain a trained depression detection model.

[0020] A further preferred technical solution of the present invention is that the collected voice data is preprocessed in step S100 to construct a voice data set; specifically:

[0021] The collected speech data was normalized, the sampling rate was adjusted to a unified standard of 16 kHz, and the speech signal was processed using a non-overlapping sliding window, with each segment length being 10 seconds.

[0022] The complete speech signal of the i-th subject is defined as ,in , represents the total length of speech, and I is the total number of subjects; After sliding window processing, it is divided into a set of several fragments , the label of each segment Labels with original speech Same, where J represents the total number of segments; each segment The length is ,Label Indicates whether the segment belongs to depressive speech, the predicted label of the speech Obtained by majority voting over all segment labels.

[0023] Preferably, the learnable frequency domain filter bank module in step S200 selects spectral features related to depression from the speech data by adaptively adjusting the frequency response of the filter; specifically:

[0024] S210: The voice segment Perform Fourier transform and convert it into the corresponding Fourier spectrum ;

[0025] Introducing learnable frequency domain filter banks Perform frequency domain filtering, where the center frequency and bandwidth of a single filter are learnable parameters that are used to adaptively adjust to respond to different speech features to obtain features. , expressed as:

[0026] ;

[0027] in is the filter index, is the number of filters, ⊙ is the stationary point multiplication operation; the symbol Represents the mean square absolute value operation, which is used to emphasize the amplitude information in the filter; is defined as:

[0028] ;

[0029] Where 𝑏 represents the subscript of the subband bin; and represent the learnable center frequency and standard deviation respectively;

[0030] All filters are evenly distributed in all subband sets of length 𝐵, and the length of each filter is ; In addition, the center frequency of the adjacent filter and The difference between Constrained to be non-negative; learnable frequency domain filter banks By stacking a series of Establish; for a given STFT spectrum, the output feature of the learnable frequency domain filter bank is expressed as:

[0031] ;

[0032] Among them, each Both serve as aggregation functions of the power spectrum, and retain or attenuate relevant frequency information by adjusting its importance weight; Represents an event frame;

[0033] S220, each element of the output feature Along the filter axis, by using the exponential average of past values Normalized, expressed as:

[0034] ;

[0035] The value of is updated as follows:

[0036] ;

[0037] After normalization, the compression operation is used to reduce The characteristic dynamic range of is expressed as:

[0038] ;

[0039] in, represents the timeframe range, is a small constant; 、 、 and They are gain, exponent, offset and smoothing coefficient, which are parameterized together and used as learnable parameters of the learnable frequency domain filter bank module for optimization and adjustment.

[0040] Preferably, the hierarchical speech feature extraction module in step S200 performs block processing on the input spectral features and performs compression and sparsification operations through forward propagation, gradually discarding features irrelevant to the depression detection task and retaining key depression-related features; specifically:

[0041] S230, assuming that speech representations are distributed in The union of low-dimensional subspaces, these subspaces are connected by orthogonal basis sets. Projection, where , represents the dimension of the token, represents the dimension of the subspace after projection;

[0042] Define the hierarchical speech feature extraction module to maximize the following objective function:

[0043] ;

[0044] in, Represents the mapping function of each layer of the hierarchical speech feature extraction module, represents the parameter space of the mapping function, Indicates the expected value, and denote the coding rates between and within representations, respectively; represents the number of layers in the network, is by The tokenized representation obtained by inputting into the patch embedding module based on the fully connected layer, specifically, It is divided into multiple patch blocks, and the dimension of each patch block is , is the dimension of each patch block after the embedding module. Subsequently, a fully connected patch embedding layer serializes these blocks, where , represents a classification token, Represents the 0th layer, that is, the 1st to Pth input tokens when first input;

[0045] The hierarchical speech feature extraction module establishes a two-stage optimization network structure for the final representation, where each stage contains an incremental operation process based on its input distribution, perform forward mapping ;

[0046] For a given subspace projection matrix , the hierarchical speech feature extraction module first minimizes the objective function During the gradient descent iteration, the multi-head attention mechanism is used to transfer the token Compress into the specified subspace to obtain a compressed representation , expressed as:

[0047] ;

[0048] in, Represents the learning rate, and the multi-head attention mechanism MSA includes A self-attention mechanism, ;

[0049] ;

[0050] ;

[0051] in, represents the gradient, is the total number of patch blocks, is a small constant;

[0052] S240, the hierarchical speech feature extraction module maximizes the , for token Perform sparsification to obtain an output representation with sparseness and structure , the strategy for sparse processing is:

[0053] 1) Introducing a learnable sparse dictionary Achieve a trade-off between representation diversity and sparsity, i.e. ;

[0054] 2) Token representation Introduction Norm constraint to enhance its sparsity;

[0055] Based on these two strategies, redefine for:

[0056] ;

[0057] in, is the regularization term for sparsification;

[0058] For the first item, when the dictionary When approximately orthogonal, ,but:

[0059] ;

[0060] Accordingly, Solve according to the following formula:

[0061] ;

[0062] Relax the above formula to a formula with learnable reweights The convex optimization problem of the norm is then transformed into a joint solution dictionary and weight parameters The problem of enhancing the sparsity of representation can be expressed as:

[0063] ;

[0064] in, is the weight used to retain the critical subspace;

[0065] The iterative shrinkage-thresholding algorithm is used in the optimization process, and the iterative update is as follows:

[0066] ;

[0067] in, represents the step length, It is used to introduce non-negativity constraints. By stacking multiple hierarchical speech feature extraction modules, the representation will be gradually compressed and sparse to extract more discriminative features.

[0068] As a preference, in the pre-training phase of step S300, the final is input to the fully connected layer FC, The classification task is completed in , and all parameters are optimized by back propagation cross entropy loss function Make updates;

[0069] ;

[0070] in, , Map the [CLS] token to a logits vector for classification prediction.

[0071] Preferably, in the fine-tuning stage in step S300, the depression screening network framework is initialized with the network weights in the pre-training stage, and a self-supervised learning strategy is adopted to fine-tune the model parameters of the depression screening network framework on a labeled speech dataset using known labels to obtain a trained depression detection model; specifically,

[0072] Extracting non-overlapping segments from the same speech signal and , generate similar input pairs from the same speech signal, where 𝑚 and 𝑛 represent the indices of each segment;

[0073] A network framework consisting of two encoders, a projector, a predictor, and a discriminator is constructed. Both encoders include a learnable frequency-domain filter bank module and a hierarchical speech feature extraction module, which are used to extract different emotion representations (ER) and speaker representations (SR), respectively.

[0074] The fragment and Input learnable frequency domain filter bank module to extract features and ,in ; These features are further processed by the hierarchical speech feature extraction module, and the compression and sparsity operations are used to extract and ; In each fragment and Interact and reconstruct to generate and Then, through the projector and Perform nonlinear transformation to obtain and ; Finally, and Input predictor, get and , decouple emotion representation ER and speaker representation SR, and use emotion representation ER to initialize the depression screening network framework;

[0075] The fine-tuning stage uses a combination of multiple loss functions, including the negative cosine similarity loss function , discriminant constraint , local consistency constraints and adversarial loss function ; The fine-tuning phase is performed by minimizing the following total loss function:

[0076] ;

[0077] in, 、 and is a hyperparameter.

[0078] Another aspect of the present invention provides a depression detection system, comprising:

[0079] A model construction module, configured to apply the above-mentioned speech-based depression detection model construction method to construct a trained depression detection model;

[0080] The depression detection module is used to collect the test subject's voice data, and after preprocessing, input it into the trained depression detection model to extract depression-related features. The classifier is then used to detect depression and determine the specific category.

[0081] The visualization display module is used to visualize the classification results and depression biomarkers, that is, the features learned by the depression detection model.

[0082] Another aspect of the present invention provides a non-transitory computer-readable storage medium having computer instructions stored thereon, which enable a computer to execute the above-mentioned method for constructing a speech-based depression detection model.

[0083] Another aspect of the present invention provides an electronic device comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus, and the processor calls logic instructions in the memory to execute the above-mentioned method for constructing a speech-based depression detection model.

[0084] Another aspect of the present invention provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer executes the above-mentioned speech-based depression detection model construction method.

[0085] Beneficial Effects: This paper constructs a highly efficient deep learning screening network model (Learning Interpretable Depression Representations in Speech, DisNet) for depression detection using speech. This model can learn depressive features from speech end-to-end, while also quantifying and visualizing depression biomarkers. Furthermore, to address the weak interpretability of the model due to limited clinical data, this paper employs a novel self-supervised learning strategy for speech (SLRD) to improve feature interpretability and model generalization.

[0086] Through the synergistic effect of the learnable frequency domain filter bank module and the hierarchical speech feature extraction module, DisNet in the present invention can more effectively extract depression-related features in speech, achieve better classification performance than existing methods, and improve the accuracy of depression detection; compared with traditional deep learning methods, DisNet in the present invention can clearly reveal the acoustic features related to depression in speech, provide clinicians with intuitive diagnostic basis, and enhance the interpretability of the model; the speech self-supervised learning strategy SLRD strategy of the present invention can alleviate the problem of limited labeled data to a certain extent, improve the generalization ability of the model and the recognition ability of depression features, and reduce the dependence on labeled data. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 This is an overall flow chart of the method for constructing a speech-based depression detection model of the present invention;

[0088] Figure 2 This is a data processing flow chart of a learnable frequency domain filter bank module (LFB) in Example 1 of the present invention;

[0089] Figure 3 This is a data processing flow chart of the hierarchical speech feature extraction module (HRE) in Example 1 of the present invention;

[0090] Figure 4 Flowchart of the self-supervised learning strategy (SLRD) in Example 1 of the present invention;

[0091] Figure 5 This is a diagram showing the differential biomarker recognition results in Example 2 of the present invention;

[0092] Figure 6 This is a visualization diagram of the LFB characteristics of the healthy control group in Example 2 of the present invention;

[0093] Figure 7 This is a visualization diagram of the LFB characteristics of the depression group in Example 2 of the present invention. DETAILED DESCRIPTION

[0094] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the embodiments described are part of the embodiments of the present invention, not all of the embodiments, and they should not be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0095] Based on the problems existing in the background technology, we assume that, in the SDD task, the acoustic signatures in pronunciation are independent of the text content, while the identified local patterns are located in specific frequency bands within a specific time period in the spectral features. These frequency bands can serve as biomarkers to help detect depression.

[0096] The principle behind this invention is: We assume that in the SDD task, the acoustic signatures in pronunciation are independent of the text content, while the identified local patterns are located in specific frequency bands within a specific time period in the spectral features. These frequency bands can serve as biomarkers to help detect depression. To investigate this, an interpretable depression screening network (DisNet) can be designed to target regions containing these depression biomarkers. Figure 1-Figure 7 The present invention describes a method for constructing a speech-based depression detection model and a depression detection system.

[0097] Example 1: This example provides a method for constructing a speech-based depression detection model, comprising the following steps:

[0098] S100: Collect voice data and add labels to indicate whether the voice is depressive; pre-process the collected voice data to construct a voice dataset.

[0099] The collected speech data was normalized, the sampling rate was adjusted to a unified standard of 16 kHz, and the speech signal was processed using a non-overlapping sliding window, with each segment length being 10 seconds.

[0100] The complete speech signal of the i-th subject is defined as ,in , represents the total length of speech, and I is the total number of subjects; After sliding window processing, it is divided into a set of several fragments , the label of each segment Labels with original speech Same, where J represents the total number of segments; each segment The length is ,Label Indicates whether the segment belongs to depressive speech, the predicted label of the speech Obtained by majority voting over all segment labels.

[0101] S200, build a depression screening network framework, such as Figure 1 As shown in the figure, this is a joint optimization framework designed specifically for the SDD (Depression Speech Detection) task. It primarily involves the construction and pre-training of the DisNet component, as well as fine-tuning using a self-supervised learning strategy. First, to obtain spectral features associated with depression, a learnable frequency-domain filter bank (LFB) module recalibrates the importance of each frequency in speech. Next, a hierarchical speech feature extraction (HRE) module compresses and sparsifies the learned features to extract depression-related representations and locate key articulatory regions. Finally, a self-supervised learning strategy, SLRD, guides the DisNet to focus on emotion representations, providing a general and interpretable representation for the fine-tuning phase of the SDD task.

[0102] It can be seen that DisNet consists of two core modules: the learnable frequency-domain filterbank (LFB) module and the hierarchical speech representation extraction (HRE) module.

[0103] The method of processing speech data by the frequency domain filter bank module (LFB) can be learned, such as Figure 2 As shown in Figure 2, this module selects frequency band features associated with depression from speech signals by adaptively adjusting the filter's frequency response. The LFB module consists of a learnable frequency domain filter and a nonlinear transform, enabling dynamic frequency response adjustment and parameter updates. The LFB module is designed to extract spectral features associated with depression from speech signals. Its core concept is to dynamically adjust the frequency response using a learnable filter bank to selectively extract frequency band information associated with depression.

[0104] LFB selects spectral features related to depression from speech data by adaptively adjusting the frequency response of the filter; the specific method is:

[0105] S210. It is known that the convolution theorem provides time domain and frequency domain methods for signal analysis as follows: ;

[0106] in, represents the Fourier transform, and ⊙ denote convolution and stationary-point multiplication operations. and is the original time series signal and its corresponding Fourier spectrum. and The impulse response and frequency response of the corresponding filter. In view of this, the existing time domain filtering method is parameterized The center frequency and bandwidth are set with a step size of and kernel size The convolutional layer is used to approximate However, obtaining the output characteristics of the time-domain filter bank requires more computational cost, which is calculated as follows:

[0107] ;

[0108] It is worth noting that the equivalent calculation in the frequency domain only requires one Fourier transform.

[0109] Therefore, in this embodiment, the voice segment Perform Fourier transform and convert it into the corresponding Fourier spectrum ;

[0110] Then, a more efficient approach is adopted to introduce learnable frequency domain filter banks Perform frequency domain filtering, where the center frequency and bandwidth of a single filter are learnable parameters that are used to adaptively adjust to respond to different speech features to obtain features. , expressed as:

[0111] ;

[0112] in is the filter index, is the number of filters, ⊙ is the stationary point multiplication operation; the symbol Represents the mean square absolute value operation, which is used to emphasize the amplitude information in the filter; is defined as:

[0113] ;

[0114] Where 𝑏 represents the subscript of the subband bin; and represent the learnable center frequency and standard deviation respectively;

[0115] All filters are evenly distributed in all subband sets of length 𝐵, and the length of each filter is ; In addition, the center frequency of the adjacent filter and The difference between Constrained to be non-negative; learnable frequency domain filter banks By stacking a series of Establish; for a given STFT spectrum, the output feature of the learnable frequency domain filter bank is expressed as:

[0116] ;

[0117] Among them, each Both serve as aggregation functions of the power spectrum, and retain or attenuate relevant frequency information by adjusting its importance weight; Represents an event frame;

[0118] S220, nonlinear transformation: used to enhance robustness to loudness changes.

[0119] In this embodiment, a nonlinear transformation is used to Normalize and compress the feature. Specifically, each element of the output feature Along the filter axis, by using the exponential average of past values Normalized, expressed as:

[0120] ;

[0121] The value of is updated as follows:

[0122] ;

[0123] After normalization, the compression operation is used to reduce The characteristic dynamic range of is expressed as:

[0124] ;

[0125] in, represents the timeframe range, is a small constant; 、 、 and They are gain, exponent, offset and smoothing coefficient, which are parameterized together and used as learnable parameters of LFB for optimization and adjustment.

[0126] Hierarchical speech feature extraction module HRE, such as Figure 3 As shown in the figure, the input spectral features are divided into blocks, and compression and sparsification operations are performed through forward propagation, gradually discarding features that are not related to the depression detection task and retaining key depression-related features;

[0127] S230, compression operation: projecting the features into a low-dimensional subspace to compress the feature dimensions.

[0128] The inherent temporal variability of speech and the uniqueness of each segment bring challenges to identifying depression differences within the spectrum. To solve this problem, this embodiment introduces the HRE module to Decorrelation is performed to retain the key information of the SDD task ( ).

[0129] Assume that speech representations are distributed across The union of low-dimensional subspaces, these subspaces are connected by orthogonal basis sets. Projection, where , represents the dimension of the token, represents the dimension of the projected subspace; Figure 3 As shown, the HRE module describes the token The marginal distribution of .

[0130] To ensure the independence between representations and the compressibility within the same representation, HRE maximizes the following objective function:

[0131] ;

[0132] in, Represents the mapping function of each layer of the hierarchical speech feature extraction module, represents the parameter space of the mapping function, Indicates the expected value, and denote the coding rates between and within representations, respectively; represents the number of layers in the network, is by The tokenized representation obtained by inputting into the patch embedding module based on the fully connected layer, specifically, It is divided into multiple patch blocks, and the dimension of each patch block is , is the dimension of each patch block after the embedding module. Subsequently, a fully connected patch embedding layer serializes these blocks, where , represents a classification token, Represents the 0th layer, that is, the 1st to Pth input tokens when first input;

[0133] HRE establishes a two-stage optimization network structure for the final representation, where each stage contains an incremental operation process based on its input distribution, perform forward mapping ;

[0134] For a given subspace projection matrix , HRE first minimizes the objective function During the gradient descent iteration, the token is transferred to Compress into the specified subspace to obtain a compressed representation , expressed as:

[0135] ;

[0136] in, Represents the learning rate, and the multi-head attention mechanism MSA includes A self-attention mechanism, ;

[0137] ;

[0138] ;

[0139] in, represents the gradient, is the total number of patch blocks, is a small constant;

[0140] S240, sparse operation: The features are sparsely processed under the action of the global dictionary and the correlation matrix to obtain a sparse and structured representation.

[0141] HRE is maximized via a relaxed proximal gradient descent step , for token Perform sparsification to obtain an output representation with sparseness and structure In this embodiment, the strategy for sparse processing is:

[0142] 1) Introducing a learnable sparse dictionary Achieve a trade-off between representation diversity and sparsity, i.e. ;

[0143] 2) Token representation Introduction Norm constraint to enhance its sparsity;

[0144] Based on these two strategies, redefine for:

[0145] ;

[0146] in, is the regularization term for sparsification;

[0147] For the first item, when the dictionary When approximately orthogonal, ,but:

[0148] ;

[0149] Accordingly, Solve according to the following formula:

[0150] ;

[0151] Different from static mesh data processing methods, our method utilizes learnable parameters from a global perspective. To estimate the temporal correlation between the projection subspaces, and thus more effectively separate the representations related to depression. Therefore, the above formula is relaxed to a model with learnable reweighting The convex optimization problem of the norm is then transformed into a joint solution dictionary and weight parameters The problem of enhancing the sparsity of representation can be expressed as:

[0152] ;

[0153] in, is the weight used to retain the critical subspace; during the optimization process, our method reduces the critical subspace The value is increased and the weight of non-critical subspace is increased, thereby focusing on important subspace.

[0154] In addition, the parameter optimization process uses the Iterative Shrinkage-Thresholding Algorithm (ISTA), and the iterative update is as follows:

[0155] ;

[0156] in, represents the step length, Used to introduce non-negativity constraints. By stacking multiple HREs, the representation will be gradually compressed and sparsified to extract more discriminative features.

[0157] S300, conduct two-stage training on the constructed depression screening network framework;

[0158] The first stage is the pre-training stage. It uses an unsupervised learning strategy to randomly sample speech samples from the speech dataset and extract non-overlapping segments as input data for the depression screening network framework. The sampled segments are input into a learnable frequency domain filter bank module and a hierarchical speech feature extraction module to extract emotion representation and speaker representation. The extracted representations are then interacted and reconstructed. The model parameters of the depression screening network framework are updated according to the designed loss function, and the weights of the depression screening network framework are saved.

[0159] Pre-training phase, finally is input to the fully connected layer FC, The classification task is completed in , and all parameters are optimized by back propagation cross entropy loss function Make updates;

[0160] ;

[0161] in, , Map the [CLS] token to a logits vector for classification prediction.

[0162] The second stage is the fine-tuning stage. The network weights in the pre-training stage are used to initialize the depression screening network framework. A self-supervised learning strategy is adopted. On a labeled speech dataset, the model parameters of the depression screening network framework are fine-tuned using known labels to obtain a trained depression detection model.

[0163] In order to mitigate the impact of limited labeled data and improve the reliability of biomarkers, this embodiment designs a SLRD strategy. SLRD is a self-supervised learning strategy through speech representation separation. Specifically, SLRD does not consider text content and assumes that language segments extracted from the same speech will approximately express consistent emotional states in a short period of time. Therefore, the emotion representation (ER) and speaker representation (SR) extracted from different segments of the same speech are similar or even identical. We recombined the ER with the SR in each segment to promote the interaction and reconstruction of the representations. Finally, by decoupling ER and SR and initializing DisNet with ER, the model can be guided to focus on the emotional area of ​​interest in advance, thereby improving the interpretability of the learned depression representation. Specifically:

[0164] Extracting non-overlapping segments from the same speech signal and , generate similar input pairs from the same speech signal, where 𝑚 and 𝑛 represent the indices of each segment;

[0165] A network framework consisting of two encoders, a projector, a predictor, and a discriminator is constructed. Both encoders include a learnable frequency-domain filter bank module and a hierarchical speech feature extraction module, which are used to extract different emotion representations (ER) and speaker representations (SR), respectively.

[0166] The fragment and Input learnable frequency domain filter bank module to extract features and ,in ; These features are further processed by the hierarchical speech feature extraction module, and the compression and sparsity operations are used to extract and ; In each fragment and Interact and reconstruct to generate and Then, through the projector and Perform nonlinear transformation to obtain and ; Finally, and Input predictor, get and , decouple emotion representation ER and speaker representation SR, and use emotion representation ER to initialize the depression screening network framework;

[0167] 、 、 、 、 、 、 、 、

[0168] 、 、 、 、 、 、 、 、 、 、 、

[0169] The fine-tuning stage uses a combination of multiple loss functions, including the negative cosine similarity loss function , discriminant constraint , local consistency constraints and adversarial loss function ; The fine-tuning phase is performed by minimizing the following total loss function:

[0170] ;

[0171] in, 、 and is a hyperparameter.

[0172] First, negative cosine similarity is selected as the loss function to measure the similarity between representations, which is expressed as:

[0173] ;

[0174] in represents the stopped gradient operation. Therefore, the similarity between two speech segments is only determined by the same emotion representations (ERs) and speaker representations (SRs) in their latent representations.

[0175] Secondly, in order to ensure the distinction between speaker representations (SRs) and emotion representations (ERs), a discriminative constraint term is introduced. , to force the speaker representation to meet the orthogonality condition, expressed as:

[0176] ;

[0177] in is the square of the Frobenius norm.

[0178] In addition, in order to minimize the difference in emotion representations (ERs) between two speech segments, a local consistency constraint is introduced , to align them in the shared subspace. This constraint uses the Central Moment Discrepancy (CMD) to measure the difference, which is defined as follows:

[0179] ;

[0180] in, is the separation distance constant, Expressing experience expectations, It is The central moment of the sample.

[0181] In addition, in order to strengthen the global consistency of emotion representations (ERs), an adversarial loss is introduced to constrain the ER distribution of all samples to be close to the Gaussian distribution. Specifically, the discriminator uses the Earth Moving Distance (EMD) to reduce the difference between the ER distribution and the Gaussian distribution. Under the condition of satisfying Lipschitz continuity, the loss function of the discriminator is defined as follows:

[0182] ;

[0183] in represents the function of the discriminator, and the loss added to the generator DisNet is .

[0184] Example 2: A depression detection system, comprising:

[0185] A model construction module, configured to apply the speech-based depression detection model construction method of Example 1 to construct a trained depression detection model;

[0186] The depression detection module is used to collect the test subject's voice data, and after preprocessing, input it into the trained depression detection model to extract depression-related features. The classifier is then used to detect depression and determine the specific category.

[0187] To verify the validity and accuracy of the test, this example designed a healthy control group and a depression group, and conducted separate sampling. During sampling, emotional well-being and sleep patterns were assessed using the Depression Anxiety and Stress Scales (DASS-21) and the Insomnia Severity Index (ISI). Risky behaviors were assessed using the Ottawa Self-injury 2 Inventory (OSI).

[0188] Participants whose scores exceeded predefined thresholds: depression score (DS) > 9, anxiety score (AS) > 7, or stress score (SS) > 14 were labeled as having sub-healthy mental health, and the 7-item score of each subscale was multiplied by 2 to obtain the final score. In this example, the severity of depression defined by the DASS-21 score was categorized as follows: mild (10-13), moderate (14-20), severe (21-27), and very severe (≥28). For the NCs group, the DASS-21 criteria were defined as follows: depression score ≤ 9, anxiety score ≤ 7, stress score ≤ 14, ISI score ≤ 7, and no suicidal behavior on the OSI result.

[0189] The visualization display module is used to visualize the classification results and depression biomarkers, that is, the features learned by the depression detection model.

[0190] like Figure 5As shown in the figure, DisNet found specific differences in pronunciation phonemes between the control group and the depression group, and pointed out the specific range of each category. By measuring the specific range of pronunciation phonemes of the subject, it is possible to identify whether he or she suffers from depression.

[0191] like Figure 6 、 Figure 7 As shown in the figure, by visualizing the features learned by LFB, it is possible to effectively distinguish patients with depression. Discontinuities and a loss of fundamental frequency (F0) are prominent features of depressive speech. In contrast, the features of normal controls (NCs) exhibit a distinct striped distribution in the F0 region, with even energy distribution and greater continuity.

[0192] Verification results demonstrate that the present invention's technical solution can support the use of intuitive visualization methods to replace traditional questionnaires. Compared with traditional deep learning methods, DisNet can clearly reveal the acoustic features in speech that are related to depression, providing clinicians with intuitive diagnostic evidence.

[0193] Embodiment 2: This embodiment provides a non-transitory computer-readable storage medium having computer instructions stored thereon. The computer instructions enable a computer to execute a method for constructing a speech-based depression detection model. The method includes the following steps:

[0194] S100, collecting voice data and adding labels to indicate whether the voice is depressive; preprocessing the collected voice data to construct a voice dataset;

[0195] S200, constructing a depression screening network framework, wherein the depression screening network framework includes at least a learnable frequency domain filter bank module for extracting spectral features related to depression from speech signals, and a hierarchical speech feature extraction module for extracting depression-related representations and locating key pronunciation areas;

[0196] The speech data is input into the depression screening network framework, and the learnable frequency domain filter bank module selects spectral features related to depression from the speech data by adaptively adjusting the frequency response of the filter;

[0197] Input the spectral features extracted by the learnable frequency domain filter bank module into the hierarchical speech feature extraction module;

[0198] The hierarchical speech feature extraction module processes the input spectral features in blocks and performs compression and sparsification operations through forward propagation, gradually discarding features irrelevant to the depression detection task and retaining key depression-related features;

[0199] S300, conduct two-stage training on the constructed depression screening network framework;

[0200] The first stage is the pre-training stage. It uses an unsupervised learning strategy to randomly sample speech samples from the speech dataset and extract non-overlapping segments as input data for the depression screening network framework. The sampled segments are input into a learnable frequency domain filter bank module and a hierarchical speech feature extraction module to extract emotion representation and speaker representation. The extracted representations are then interacted and reconstructed. The model parameters of the depression screening network framework are updated according to the designed loss function, and the weights of the depression screening network framework are saved.

[0201] The second stage is the fine-tuning stage. The network weights in the pre-training stage are used to initialize the depression screening network framework. A self-supervised learning strategy is adopted. On a labeled speech dataset, the model parameters of the depression screening network framework are fine-tuned using known labels to obtain a trained depression detection model.

[0202] Example 3: This embodiment provides an electronic device, which may include: a processor, a communications interface, a memory, and a communications bus, wherein the processor, the communications interface, and the memory communicate with each other via the communications bus. The processor may invoke logic instructions in the memory to execute a method for constructing a speech-based depression detection model, which includes the following steps:

[0203] S100, collecting voice data and adding labels to indicate whether the voice is depressive; preprocessing the collected voice data to construct a voice dataset;

[0204] S200, constructing a depression screening network framework, wherein the depression screening network framework includes at least a learnable frequency domain filter bank module for extracting spectral features related to depression from speech signals, and a hierarchical speech feature extraction module for extracting depression-related representations and locating key pronunciation areas;

[0205] The speech data is input into the depression screening network framework, and the learnable frequency domain filter bank module selects spectral features related to depression from the speech data by adaptively adjusting the frequency response of the filter;

[0206] Input the spectral features extracted by the learnable frequency domain filter bank module into the hierarchical speech feature extraction module;

[0207] The hierarchical speech feature extraction module processes the input spectral features in blocks and performs compression and sparsification operations through forward propagation, gradually discarding features irrelevant to the depression detection task and retaining key depression-related features;

[0208] S300, conduct two-stage training on the constructed depression screening network framework;

[0209] The first stage is the pre-training stage. It uses an unsupervised learning strategy to randomly sample speech samples from the speech dataset and extract non-overlapping segments as input data for the depression screening network framework. The sampled segments are input into a learnable frequency domain filter bank module and a hierarchical speech feature extraction module to extract emotion representation and speaker representation. The extracted representations are then interacted and reconstructed. The model parameters of the depression screening network framework are updated according to the designed loss function, and the weights of the depression screening network framework are saved.

[0210] The second stage is the fine-tuning stage. The network weights in the pre-training stage are used to initialize the depression screening network framework. A self-supervised learning strategy is adopted. On a labeled speech dataset, the model parameters of the depression screening network framework are fine-tuned using known labels to obtain a trained depression detection model.

[0211] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0212] Embodiment 4: This embodiment provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform a method for constructing a speech-based depression detection model. The method includes the following steps:

[0213] S100, collecting voice data and adding labels to indicate whether the voice is depressive; preprocessing the collected voice data to construct a voice dataset;

[0214] S200, constructing a depression screening network framework, wherein the depression screening network framework includes at least a learnable frequency domain filter bank module for extracting spectral features related to depression from speech signals, and a hierarchical speech feature extraction module for extracting depression-related representations and locating key pronunciation areas;

[0215] The speech data is input into the depression screening network framework, and the learnable frequency domain filter bank module selects spectral features related to depression from the speech data by adaptively adjusting the frequency response of the filter;

[0216] Input the spectral features extracted by the learnable frequency domain filter bank module into the hierarchical speech feature extraction module;

[0217] The hierarchical speech feature extraction module processes the input spectral features in blocks and performs compression and sparsification operations through forward propagation, gradually discarding features irrelevant to the depression detection task and retaining key depression-related features;

[0218] S300, conduct two-stage training on the constructed depression screening network framework;

[0219] The first stage is the pre-training stage. It uses an unsupervised learning strategy to randomly sample speech samples from the speech dataset and extract non-overlapping segments as input data for the depression screening network framework. The sampled segments are input into a learnable frequency domain filter bank module and a hierarchical speech feature extraction module to extract emotion representation and speaker representation. The extracted representations are then interacted and reconstructed. The model parameters of the depression screening network framework are updated according to the designed loss function, and the weights of the depression screening network framework are saved.

[0220] The second stage is the fine-tuning stage. The network weights in the pre-training stage are used to initialize the depression screening network framework. A self-supervised learning strategy is adopted. On a labeled speech dataset, the model parameters of the depression screening network framework are fine-tuned using known labels to obtain a trained depression detection model.

[0221] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0222] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0223] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for constructing a speech-based depression detection model, characterized in that: The steps include: S100, collecting voice data and adding labels to indicate whether the voice is depressive; preprocessing the collected voice data to construct a voice dataset; S200, constructing a depression screening network framework, wherein the depression screening network framework includes at least a learnable frequency domain filter bank module for extracting spectral features related to depression from speech signals, and a hierarchical speech feature extraction module for extracting depression-related representations and locating key pronunciation areas; The speech data is input into the depression screening network framework, and the learnable frequency domain filter bank module selects spectral features related to depression from the speech data by adaptively adjusting the frequency response of the filter; Input the spectral features extracted by the learnable frequency domain filter bank module into the hierarchical speech feature extraction module; The hierarchical speech feature extraction module processes the input spectral features in blocks and performs compression and sparsification operations through forward propagation, gradually discarding features irrelevant to the depression detection task and retaining key depression-related features; S300, conduct two-stage training on the constructed depression screening network framework; The first stage is the pre-training stage. It uses an unsupervised learning strategy to randomly sample speech samples from the speech dataset and extract non-overlapping segments as input data for the depression screening network framework. The sampled segments are input into a learnable frequency domain filter bank module and a hierarchical speech feature extraction module to extract emotion representation and speaker representation. The extracted representations are then interacted and reconstructed. The model parameters of the depression screening network framework are updated according to the designed loss function, and the weights of the depression screening network framework are saved. The second stage is the fine-tuning stage. The network weights in the pre-training stage are used to initialize the depression screening network framework. A self-supervised learning strategy is adopted. On a labeled speech dataset, the model parameters of the depression screening network framework are fine-tuned using known labels to obtain a trained depression detection model.

2. The method for constructing a speech-based depression detection model according to claim 1, wherein: Step S100 pre-processes the collected voice data to construct a voice data set; specifically: The collected speech data was normalized, the sampling rate was adjusted to a unified standard of 16 kHz, and the speech signal was processed using a non-overlapping sliding window, with each segment length being 10 seconds. The complete speech signal of the i-th subject is defined as ,in , represents the total length of speech, and I is the total number of subjects; After sliding window processing, it is divided into a set of several fragments , the label of each segment Labels with original speech Same, where J represents the total number of segments; each segment The length is ,Label Indicates whether the segment belongs to depressive speech, the predicted label of the speech Obtained by majority voting over all segment labels.

3. The method for constructing a speech-based depression detection model according to claim 2, wherein: The learnable frequency domain filter bank module in step S200 selects spectral features related to depression from the speech data by adaptively adjusting the frequency response of the filter; specifically: S210: The voice segment Perform Fourier transform and convert it into the corresponding Fourier spectrum ; Introducing learnable frequency domain filter banks Perform frequency domain filtering, where the center frequency and bandwidth of a single filter are learnable parameters that are used to adaptively adjust to respond to different speech features to obtain features. , expressed as: ； in is the filter index, is the number of filters, ⊙ is the stationary point multiplication operation; the symbol Represents the mean square absolute value operation, which is used to emphasize the amplitude information in the filter; is defined as: ； Where 𝑏 represents the subscript of the subband bin; and represent the learnable center frequency and standard deviation respectively; All filters are evenly distributed in all subband sets of length 𝐵, and the length of each filter is ; In addition, the center frequency of the adjacent filter and The difference between Constrained to be non-negative; learnable frequency domain filter banks By stacking a series of Establish; for a given STFT spectrum, the output feature of the learnable frequency domain filter bank is expressed as: ； Among them, each Both serve as aggregation functions of the power spectrum, and retain or attenuate relevant frequency information by adjusting its importance weight; Represents an event frame; S220, output each element of the feature Along the filter axis, by using the exponential average of past values Normalized, expressed as: ； The value of is updated as follows: ； After normalization, the compression operation is used to reduce The characteristic dynamic range of is expressed as: ； in, represents the timeframe range, is a small constant; 、 、 and They are gain, exponent, offset and smoothing coefficient, which are parameterized together and used as learnable parameters of the learnable frequency domain filter bank module for optimization and adjustment.

4. The method for constructing a speech-based depression detection model according to claim 3, wherein: The hierarchical speech feature extraction module in step S200 processes the input spectral features in blocks and performs compression and sparsification operations through forward propagation, gradually discarding features irrelevant to the depression detection task and retaining key depression-related features; Specifically: S230, assuming that speech representations are distributed in The union of low-dimensional subspaces, these subspaces are connected by orthogonal basis sets. Projection, where , represents the dimension of the token, represents the dimension of the subspace after projection; Define the hierarchical speech feature extraction module to maximize the following objective function: ； in, Represents the mapping function of each layer of the hierarchical speech feature extraction module, represents the parameter space of the mapping function, Indicates the expected value, and denote the coding rates between and within representations, respectively; represents the number of layers in the network, is by The tokenized representation obtained by inputting into the patch embedding module based on the fully connected layer, specifically, It is divided into multiple patch blocks, and the dimension of each patch block is , is the dimension of each patch block after the embedding module. Subsequently, a fully connected patch embedding layer serializes these blocks, where , represents a classification token, Represents the 0th layer, that is, the 1st to Pth input tokens when first input; The hierarchical speech feature extraction module establishes a two-stage optimization network structure for the final representation, where each stage contains an incremental operation process based on its input distribution, perform forward mapping ; For a given subspace projection matrix , the hierarchical speech feature extraction module first minimizes the objective function During the gradient descent iteration, the multi-head attention mechanism is used to transfer the token Compress into the specified subspace to obtain a compressed representation , expressed as: ； in, Represents the learning rate, and the multi-head attention mechanism MSA includes A self-attention mechanism, ; ； ； in, represents the gradient, is the total number of patch blocks, is a small constant; S240, the hierarchical speech feature extraction module maximizes the , for token Perform sparsification to obtain an output representation with sparseness and structure , the strategy for sparse processing is: 1) Introducing a learnable sparse dictionary Achieve a trade-off between representation diversity and sparsity, i.e. ; 2) Token representation Introduction Norm constraint to enhance its sparsity; Based on these two strategies, redefine for: ； in, is the regularization term for sparsification; For the first item, when the dictionary When approximately orthogonal, ,but: ； Accordingly, Solve according to the following formula: ； Relax the above formula to a formula with learnable reweights The convex optimization problem of the norm is then transformed into a joint solution dictionary and weight parameters The problem of enhancing the sparsity of representation can be expressed as: ； in, is the weight used to retain the critical subspace; The iterative shrinkage-thresholding algorithm is used in the optimization process, and the iterative update is as follows: ； in, represents the step length, It is used to introduce non-negativity constraints. By stacking multiple hierarchical speech feature extraction modules, the representation will be gradually compressed and sparse to extract more discriminative features.

5. The method for constructing a speech-based depression detection model according to claim 4, wherein: In the pre-training phase of step S300, the final is input to the fully connected layer FC, The classification task is completed in , and all parameters are optimized by back propagation cross entropy loss function Make updates; ； in, , Map the [CLS] token to a logits vector for classification prediction.

6. The method for constructing a speech-based depression detection model according to claim 2, wherein: In the fine-tuning phase in step S300, the depression screening network framework is initialized with the network weights from the pre-training phase. A self-supervised learning strategy is adopted to fine-tune the model parameters of the depression screening network framework on a labeled speech dataset using known labels to obtain a trained depression detection model. Specifically, Extracting non-overlapping segments from the same speech signal and , generate similar input pairs from the same speech signal, where 𝑚 and 𝑛 represent the indices of each segment; A network framework consisting of two encoders, a projector, a predictor, and a discriminator is constructed. Both encoders include a learnable frequency-domain filter bank module and a hierarchical speech feature extraction module, which are used to extract different emotion representations (ER) and speaker representations (SR), respectively. The fragment and Input learnable frequency domain filter bank module to extract features and ,in ; These features are further processed by the hierarchical speech feature extraction module, and the compression and sparsity operations are used to extract and ; In each fragment and Interact and reconstruct to generate and Then, through the projector and Perform nonlinear transformation to obtain and ; Finally, and Input predictor, get and , decouple emotion representation ER and speaker representation SR, and use emotion representation ER to initialize the depression screening network framework; The fine-tuning stage uses a combination of multiple loss functions, including the negative cosine similarity loss function , discriminant constraint , local consistency constraints and adversarial loss function ; The fine-tuning phase is performed by minimizing the following total loss function: ； in, 、 and is a hyperparameter.

7. A depression detection system, characterized in that: include: A model construction module, configured to apply the speech-based depression detection model construction method according to any one of claims 1 to 6 to construct a trained depression detection model; The depression detection module is used to collect the test subject's voice data, and after preprocessing, input it into the trained depression detection model to extract depression-related features. The classifier is then used to detect depression and determine the specific category. The visualization display module is used to visualize the classification results and depression biomarkers, that is, the features learned by the depression detection model.

8. A non-transitory computer-readable storage medium, characterized in that Computer instructions are stored thereon, and the computer instructions enable the computer to execute the method for constructing a speech-based depression detection model according to any one of claims 1 to 6.

9. An electronic device, characterized in that: include: A processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus, and the processor calls logic instructions in the memory to execute the speech-based depression detection model construction method described in any one of claims 1-6.

10. A computer program product, characterized in that The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer executes the speech-based depression detection model construction method according to any one of claims 1 to 6.

Images