Depression state recognition system based on combination of time domain and frequency domain

Abstract

The invention discloses a depression state recognition system based on time-frequency domain combination, and the system comprises an obtaining module which is configured to obtain a to-be-recognized heart sound signal; the preprocessing module is configured to carry out preprocessing on the heart sound signals to be recognized, and preprocessing comprises form transformation, down-sampling and filtering processing; the depression state recognition module is configured to input the preprocessed heart sound signals into a trained depression state recognition model to obtain a depression state recognition result; wherein the trained depression state recognition model is used for extracting Mel-frequency cepstral coefficient characteristics from the preprocessed heart sound signals and extracting frequency domain characteristics from the Mel-frequency cepstral coefficient characteristics; meanwhile, time domain features are extracted from the preprocessed heart sound signals; after the time domain features and the frequency domain features are fused, feature extraction is carried out on the fused features to obtain semantic features; and classifying the semantic features to obtain a depression identification result.

Description

Technical Field

[0001] The present invention relates to the technical field of depression state recognition, and in particular to a depression state recognition system based on the combination of time domain and frequency domain. Background Art

[0002] Depression is a common mental illness in the world today. Its main clinical features are significant and persistent low mood or loss of interest and pleasure. Its patients often have low mood, loss of interest, and withdrawn personality. Deeply depressed patients may even engage in dangerous behaviors that endanger their life safety, such as self-abuse and suicide, which has a serious negative impact on the development of individuals, families, and society.

[0003] Timely diagnosis is a prerequisite for effective treatment, but the current accurate diagnosis method for depression is not yet mature. At present, the clinical diagnosis of depression is mainly based on the depression diagnostic criteria in ICD-0 or DSM-V, combined with the patient's interview, scales, and doctor's diagnosis and treatment experience. Misdiagnosis is easily caused by subjective factors such as the patient's degree of cooperation and the doctor's proficiency. In addition, the lack of patient medical awareness and early screening tools has led to some patients having reached severe depressive disorder during diagnosis and treatment.

[0004] According to psychiatric theory, depression is an affective mental disorder with a relatively complex pathogenesis and unclear specific etiology. Studies have found that patients with depressive disorders have abnormal hypothalamic-pituitary-adrenal axis (HPA) function, and the dysfunction of the HPA axis can be reflected by the dysfunction of the autonomic nervous system. It is specifically manifested as cardiovascular disorders such as tachycardia, and the symptoms often associated with depression, such as nausea, vomiting, palpitations, chest tightness and sweating, are all related to autonomic dysfunction, and heart sound signals can reflect these changes to a certain extent. Using biological information to assist depression recognition has become an important way to improve the accuracy of depression recognition. At present, the application of biological signals is mainly eye movement signals, facial expressions, voice signals and EEG signals, but external information such as eye movement signals, facial expressions and voice signals can make the information deceptive due to the subject's inhibition or concealment. EEG signal acquisition is not easy and the acquisition price is expensive, which is not suitable for popularization among the population. Compared with speech signals, heart sound signals will not cause inaccurate detection due to subject suppression or concealment; heart sound signals are easier to collect than EEG signals, and as physiological information of the human body, they have objective and accurate characteristics. In the prior art, feature extraction of heart sound signals includes feature extraction in the time domain and frequency domain. Commonly used time domain feature extraction methods such as autocorrelation function method, peak method and zero crossing rate method are sensitive to noise and cannot provide detailed information about frequency; commonly used frequency domain feature extraction methods such as fast Fourier transform, power spectral density method and wavelet transform have high computational complexity and do not consider the instantaneous changes of heart sound signals.

[0005] In addition, in recent years, with the development of the Internet and science and technology, deep learning has been widely used in the medical field due to its powerful data processing and data mining capabilities. When training the model, the loss function used is relatively simple and cannot be compatible with classification accuracy, sample weight and prediction probability at the same time. Summary of the invention

[0006] In order to solve the shortcomings of the prior art, the present invention provides a depression state recognition system based on the combination of time and frequency domains; clinically, depression is often accompanied by autonomic dysfunction, leading to cardiovascular disorders such as tachycardia. In order to effectively reflect these physiological changes, the present invention selects heart sound signals for the recognition of depression states. This method is simple, easy to use and highly practical.

[0007] On the one hand, a depression state recognition system based on the combination of time and frequency domains is provided, including:

[0008] An acquisition module, which is configured to: acquire a heart sound signal to be identified;

[0009] A preprocessing module, which is configured to: preprocess the heart sound signal to be recognized, wherein the preprocessing includes: form conversion, downsampling and filtering processing;

[0010] A depression state recognition module is configured to: input the preprocessed heart sound signal into the trained depression state recognition model to obtain a depression state recognition result;

[0011] Among them, the trained depression state recognition model is used to extract MFCC features from the preprocessed heart sound signal, and extract frequency domain features from the MFCC features; at the same time, time domain features are extracted from the preprocessed heart sound signal; after fusing the time domain features and the frequency domain features, feature extraction is performed on the fused features to obtain semantic features; the semantic features are classified to obtain depression recognition results.

[0012] The above technical solution has the following advantages or beneficial effects:

[0013] (1) The heart sound signal is input into a recognition model established by using a deep neural network. The method has the characteristic of obtaining results as soon as the data is input, and the recognition speed is fast. The method provided greatly reduces the subjective factors of doctors and patients, and the detection results are more objective. The heart sound signal is applied to the classification of depression status. Compared with the currently used scales in the clinical diagnosis of depression, the data of the present invention is easy to obtain and the cost is low.

[0014] (2) The Bi-LSTM neural network is used to extract the time domain features of the heart sound signal, and the Mel-frequency cepstral coefficients (MFCC) are used to extract the frequency domain features of the heart sound signal through a four-layer two-dimensional convolutional neural network (2D-CNN). The signal features are extracted from both the time domain and the frequency domain. The extracted features are more comprehensive, and the information is complementary. It can provide detailed information about the frequency and obtain the instantaneous changes of the heart sound signal, while simplifying the computational complexity. The semantic features extracted by the fused time-frequency domain features through the Transformer network architecture can express the heart sound signal more comprehensively.

[0015] (3) A total loss function formed by the weighted superposition of four loss functions is used, which comprehensively considers classification accuracy, sample weight and prediction probability, thereby improving the accuracy of depression identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0017] Figure 1 The figure is a flow chart of a method for identifying a depressive state according to an embodiment of the present invention.

[0018] Figure 24 is a flow chart of a feature extraction module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0020] Embodiment 1, as Figure 1 As shown, this embodiment provides a depression state recognition system based on the combination of time and frequency domains, including:

[0021] An acquisition module, which is configured to: acquire a heart sound signal to be identified;

[0022] A preprocessing module, which is configured to: preprocess the heart sound signal to be recognized, wherein the preprocessing includes: form conversion, downsampling and filtering processing;

[0023] A depression state recognition module is configured to: input the preprocessed heart sound signal into the trained depression state recognition model to obtain a depression state recognition result;

[0024] Among them, the trained depression state recognition model is used to extract MFCC features from the preprocessed heart sound signal, and extract frequency domain features from the MFCC features; at the same time, time domain features are extracted from the preprocessed heart sound signal; after fusing the time domain features and the frequency domain features, feature extraction is performed on the fused features to obtain semantic features; the semantic features are classified to obtain depression recognition results.

[0025] Furthermore, the obtaining of the heart sound signal to be identified is performed by collecting the heart sound signal through a heart sound collection sensor.

[0026] Exemplarily, the heart sound signal to be identified refers to the heart sound signal of the subject at rest. The resting heart sound signal of the subject is collected using a physiological signal acquisition device. During the signal acquisition process, it must be ensured that the subject is in a relatively closed environment, while avoiding interference from noise, electromagnetic signals and other factors. The RM6280C multi-channel physiological data acquisition and processing system is used to collect the heart sound signals of the subject. The instrument sampling rate is set to 1kHz, the sensitivity is 25mV / grid, the scanning speed is 200ms / grid, and the acquisition time is 330 seconds.

[0027] In order to ensure the integrity and effectiveness of the signal, the hardware filtering function of the acquisition system was turned off. The heart sound acquisition sensor uses the XJ102 heart sound transducer, whose frequency response range is 1-500Hz. The subject lies flat on the experimental table, closes his eyes, and relaxes his whole body. Place the heart sound transducer in the mitral valve auscultation area (inside the fifth intercostal space of the left clavicle midline) and fix the heart sound transducer with medical tape to prevent the sensor from rubbing against clothing and generating noise. After the preparation is completed, start recording the heart sound signal.

[0028] If there are any unexpected abnormalities in the experiment, such as the subject moving his body, taking frequent deep breaths, etc., the data collection experiment will be interrupted and the data collection experiment will be restarted after obtaining the subject's consent. After the data collection is completed, the experimental operator will measure the subject's heart rate and blood pressure, and use a computer to organize and save all the collected experimental data.

[0029] Furthermore, the form transformation is to transform the heart sound signal into a one-dimensional matrix form.

[0030] Furthermore, the filtering process uses wavelet transform to remove the baseline drift of the heart sound signal.

[0031] Exemplarily, the heart sound signal is denoised using wavelet transform, and the sym8 wavelet basis is selected to decompose and reconstruct the heart sound signal to remove the baseline drift of the heart sound signal.

[0032] The wavelet threshold denoising method is used to denoise the signal. Wavelet transform can better characterize the details of non-stationary signals, especially heart sound signals. The basic principle of wavelet threshold denoising method is to first use wavelet transform to decompose the original signal to obtain wavelet coefficients at all levels, then filter the wavelet coefficients according to the selected threshold, and then reconstruct the filtered signal based on the filtered wavelet coefficients.

[0033] The basic principle of wavelet transform is to use a basic wavelet function that can be scaled and time-shifted to convolve with the original signal, so as to obtain the frequency characteristics of the original signal at different time points. is a basic wavelet function, and a series of sub-wavelets can be generated according to the following formula:

[0034]

[0035] Among them, a is called the scale coefficient, which is a parameter that reflects the frequency, and b is called the translation coefficient, which is a parameter that reflects the time. According to this wavelet function family, the wavelet transform formula of the original signal f(t) is:

[0036]

[0037] Among them, W f(a,b) are called wavelet coefficients. Since this method requires too much computation, a power series with base 2 is generally used to discretize the scale coefficient a of the basic wavelet function. The process of using these discrete wavelets to analyze the original signal is called Discrete Wavelet Transform (DWT). If the scale parameter in the sub-wavelet increases by a multiple of 2, then the frequency value that the sub-wavelet can detect will also decrease by a multiple of 1 / 2. The spectrum corresponding to the basic wavelet is located at the high-frequency end of the spectrum of the entire signal, occupying 1 / 2 of the entire spectrum range, while the spectrum of other sub-wavelets moves to the low-frequency end of the spectrum due to the increase in scale parameters, and the frequency spectrum range they cover also decreases accordingly by multiples of 1 / 2. Ideally, the spectrum ranges of all wavelets are connected end to end and cover each other.

[0038] In the wavelet threshold denoising method, the selection of threshold and threshold function is the most important step. The threshold used in the present invention is the VisuShrink threshold, and its formula is as follows:

[0039]

[0040] Where σ represents the standard deviation of the noise, and n represents the length of the signal. Generally, the median of the wavelet coefficients representing high frequencies divided by 0.6745 is used as the estimated value of the noise standard deviation. The threshold function uses a soft threshold function. The wavelet coefficients obtained using this function have good continuity. The formula is as follows:

[0041]

[0042] W is the wavelet coefficient and Th is the threshold.

[0043] The heart sound signal is decomposed using the sym8 wavelet, which has good frequency characteristics and can retain the characteristics of the original signal as much as possible. The number of layers of wavelet decomposition is 8. Since the frequency range corresponding to the highest frequency wavelet coefficient is 500-1000Hz, and the frequency spectrum range of the heart sound signal is 20-200Hz, it is more appropriate to use the highest frequency wavelet coefficient to estimate the noise standard deviation.

[0044] Furthermore, extracting MFCC features from the preprocessed heart sound signal includes:

[0045] (1-1) cropping the preprocessed heart sound signal;

[0046] (1-2) performing frame processing on the cropped heart sound signal;

[0047] (1-3) performing windowing processing on the frame processing result;

[0048] (1-4) performing Fourier transform on the windowing processing result;

[0049] (1-5) Construct a Mel filter bank based on the Fourier transform results;

[0050] (1-6) Calculate logarithmic energy according to the Mel filter bank;

[0051] (1-7) Perform discrete cosine transform on the logarithmic energy to obtain MFCC features.

[0052] Furthermore, the (1-1) clipping the preprocessed heart sound signal includes:

[0053] The downsampled and filtered one-dimensional matrix heart sound signal is clipped using a method based on Hidden Semi-Markov Model (HSMM).

[0054] Furthermore, the step (1-2) of performing frame processing on the pre-processed heart sound signal includes:

[0055] The frame length set in the framing operation is set to 64ms and the frame shift is 25ms.

[0056] It should be understood that the heart sound signal is not stable at the macro level, but is stable at the micro level, so we need to divide the heart sound signal into several segments for processing, and each segment is called a frame.

[0057] Furthermore, the step (1-3) of performing windowing on the frame processing result includes:

[0058] Multiply each frame signal by the Hamming window to reduce spectrum leakage. The formula of the Hamming window is as follows:

[0059]

[0060] It should be understood that due to the non-periodic truncation of the signal, a spectrum leakage problem occurs in the signal spectrum within the entire frequency band.

[0061] Furthermore, the step (1-4) of performing Fourier transform on the windowing result includes:

[0062] The formula for discrete Fourier transform is as follows:

[0063]

[0064] x(n) is the input heart sound signal, and N is the length of Fourier transform.

[0065] After obtaining the Fourier transform result, calculate the power spectrum:

[0066]

[0067] It should be understood that since it is usually difficult to see the characteristics of the signal from the transformation in the time domain, it is usually converted into energy distribution in the frequency domain for observation. Different energy distributions can represent the characteristics of different heart sound signals. After multiplying the Hamming window, each frame must be subjected to Fourier transform to obtain the energy distribution in the spectrum. The frequency spectrum of each frame is obtained by fast Fourier transform after frame division and windowing. The power spectrum of the heart sound signal is obtained by taking the modulus square of the frequency spectrum of the heart sound signal.

[0068] Furthermore, the above (1-5) constructs a Mel filter bank according to the Fourier transform result, including:

[0069] The Mel scale describes the nonlinear characteristics of human ear frequency, and its purpose is to imitate the nonlinear perception of sound by the human ear, with stronger discrimination at lower frequencies and lower discrimination at higher frequencies. Its relationship with frequency can be approximated by the following formula:

[0070]

[0071] Use the transformation formula to convert the maximum and minimum frequencies of the heart sound signal into the Mel scale, and divide the range into 20 points m evenly, and then use the transformation formula to convert the 20 points m into frequency f(m). The transformation formula is as follows:

[0072] f(m)=700×(10 m / 2595 -1)

[0073] Construct a triangular bandpass filter bank:

[0074]

[0075] Among them, H m (k) represents the mth filter.

[0076] The main purpose of using a triangular bandpass filter is to smooth the spectrum, eliminate harmonics, and highlight the resonance peak of the original signal.

[0077] Furthermore, the (1-6) calculates the logarithmic energy according to the Mel filter bank, including:

[0078] The logarithmic energy is calculated based on the obtained Mel filter bank, and the formula is as follows:

[0079]

[0080] Furthermore, the (1-7) performs discrete cosine transform on the logarithmic energy to obtain MFCC features, including:

[0081] Finally, the above logarithmic energy is brought into the Discrete Cosine Transform (DCT) to obtain the MFCC coefficients. The formula of DCT is as follows:

[0082]

[0083] Wherein, F(u) represents the MFCC coefficient, and N represents the number of triangular filters. When u=0, c(u)=N-0.5, otherwise c(u)=(N / 2)-0.5.

[0084] The standard cepstrum parameters MFCC only reflect the static characteristics of the heart sound signal parameters, but the recognition of heart sound signals also requires dynamic change information between frames, such as the trajectory of MFCC over time. It has been proven that adding the trajectory change of MFCC will improve the recognition effect. The dynamic characteristics of heart sound signals can be described by the differential spectrum of these static features. The calculation formula of the differential parameters is as follows:

[0085]

[0086] Among them, d t represents the tth first-order difference, c t+1 represents the t+1th cepstral coefficient, c t-1 represents the t-1th cepstrum coefficient. Substitute the first-order difference parameters calculated by the above formula into the second-order difference parameters.

[0087] It should be understood that the MFCC feature vector describes the envelope information of the power spectrum of a frame of speech signal, but speech recognition also requires dynamic change information between frames, such as the trajectory of MFCC over time. It has been proven that adding the trajectory change of MFCC will improve the recognition effect. Therefore, the information of several frames before and after the current frame can be used to calculate the differential coefficient and the second-order differential coefficient.

[0088] Furthermore, if Figure 2 As shown, the frequency domain features are extracted from the MFCC features, including:

[0089] A two-dimensional convolutional neural network is used to extract frequency domain features from MFCC features;

[0090] The two-dimensional convolutional neural network includes: a first two-dimensional convolutional layer, a second two-dimensional convolutional layer, a third two-dimensional convolutional layer, a fourth two-dimensional convolutional layer and a maximum pooling layer which are connected in sequence.

[0091] Furthermore, the first two-dimensional convolution layer, the second two-dimensional convolution layer, the third two-dimensional convolution layer and the fourth two-dimensional convolution layer are used to extract features in the frequency domain. The number of filters in the four convolution layers increases layer by layer (32, 64, 128 and 256) to ensure the hierarchical feature extraction process. The kernel size of all convolution layers is always maintained at (3, 3) to ensure local feature capture; the maximum pooling layer is used to summarize the features, so that the features extracted by the convolution layer are summarized through the maximum pooling layer.

[0092] It should be understood that Mel Frequency Cepstral Coefficient (MFCC) is used as the frequency domain feature of the heart sound signal, and the proposed MFCC is passed through a two-dimensional convolutional neural network to extract a suitable structure of features from image data.

[0093] The 2D-CNN network is used to learn the MFCC frequency domain features of the heart sound signal. The MFCC feature is a visual representation of the frequency information of the heart sound signal. 2D-CNN is a suitable structure for extracting features from image data. The Mel-spectrogram can easily detect frequency features, and 2D-CNN can effectively learn these features.

[0094] CNN is one of the most commonly used artificial neural networks. It has translation invariance and weight sharing, so it can process large-scale data in parallel, is faster during training and testing, requires less memory for the model, and has a stabilizing effect on the extraction of data features. In addition, the CNN network avoids the complex manual feature extraction and feature screening problems in traditional machine learning methods, and can mine deeper features than manually extracted features, effectively improving the generalization ability of the model.

[0095] The convolution layer mainly performs the most basic feature extraction work, performing discrete convolution processing on the input signal and the convolution kernel to obtain different features. The size of the convolution kernel needs to be continuously optimized based on experience and the results of the network training process. The convolution operation is to move the convolution kernel by a step in the horizontal or vertical direction, and then multiply the convolution kernel by the pixel value at the corresponding position and sum them as the new feature map value corresponding to the current convolution operation. The specific operation process formula in the convolution layer is as follows:

[0096]

[0097] in and They represent the i-th input feature map of the l-1th layer and the j-th output feature map of the lth layer respectively. Represents the convolution kernel that connects the above two; Represents the bias coefficient of this layer, which can improve the fitting ability of the network; M jIndicates the connection between the input and output feature maps of this layer; * indicates discrete convolution. The convolution operation is only a linear operation and cannot express nonlinear features. In order to better mine features and enhance the network expression ability of CNN, it is necessary to add a nonlinear activation function after convolution. In the above formula, f(·) represents the activation function. Commonly used activation functions in deep learning theory include: Sigmoid function, ReLU function, Tanh function, etc., which can be flexibly selected according to the actual application scenario. The mathematical expression of the Sigmoid function is as follows:

[0098]

[0099] The output of this function is limited to between 0 and 1. It is a monotonically differentiable nonlinear function, often called an S-shaped growth curve, which can make the training process based on gradient descent more stable. However, the Sigmoid function is prone to saturation problems, that is, when the input data is very large or very small, the function gradient is close to zero, resulting in the inability to effectively update the network parameters. The Tanh function is very similar to the Sigmoid function, and its expression is as follows:

[0100]

[0101] The Tanh function is symmetric about the zero point, and its output is mapped to (-1, 1), but it also has the problem of easy saturation. The ReLU function outputs a constant gradient of 1 when x<0, which can solve the easy saturation problem of the above two activation functions; when x>0, the output of the function is fixed and always 0, which increases the sparsity of the network and thus improves the generalization ability of the network. The mathematical expression of the ReLU function is as follows:

[0102]

[0103] Since the ReLU function has a very small amount of calculation and is easy to implement in hardware, it is used as the current preferred activation function.

[0104] The pooling layer is also called the downsampling layer. It is usually used after the convolution layer to perform dimensionality reduction processing on the feature map obtained by the convolution layer, effectively reducing the amount of neural network calculations while retaining most of the feature information. The pooling operation is to perform statistical analysis on the features in the pooling area and use the statistical results to represent the features of the entire area. Commonly used pooling methods include average pooling and maximum pooling. If the average value is used to replace the feature value of the pooling area, it is an average pooling operation; if the maximum value is used to replace the feature value of the pooling area, it is a maximum pooling operation. The present invention adopts maximum pooling, and the maximum pooling operation is shown in the formula:

[0105]

[0106] Wherein, max(·) represents the maximum value operation; n represents the position of the feature map sampling point; R and S represent the pooling area and step size, respectively, and r is the index of the sampling point position in the pooling area. In this study, the pooling area and step size are set equal to avoid overlapping of the pooling sub-areas.

[0107] Furthermore, extracting time domain features from the preprocessed heart sound signal includes:

[0108] The Bi-LSTM neural network is used to extract time domain features from the preprocessed heart sound signals.

[0109] It should be understood that the LSTM network is a special recurrent neural network (RNN), which alleviates the long-term dependency problem that is common in RNN to a certain extent. LSTM can effectively transmit and express information in long time series without causing useful information from a long time ago to be ignored or forgotten. It can better handle the dependency relationship of long sequences and avoid the problem of gradient vanishing or exploding as the sequence length increases.

[0110] LSTM uses gate control structures and cell states to process information at different times. Each LSTM module includes a cell state and three gate structures: forget gate, input gate, and output gate. The cell state saves the current LSTM state information and passes it to the LSTM at the next moment. The three gates are used to select which information can be remembered and passed down, and which information needs to be forgotten. Among them:

[0111] The forget gate decides which previous memories to forget based on the new input and the output of the previous moment. The formula is as follows:

[0112] f t =σ(W f ·[h t-1 ,X t ]+b f )

[0113] where h t-1 represents the output at the previous moment, X t represents the input signal at the current moment, σ represents the sigmoid activation function layer, W f and b f Represents the training parameters and bias parameters of the forget gate.

[0114] The function of the input gate is opposite to that of the forget gate. It determines the new input signal X t and the output h at the previous moment t-1 Which information will be retained in the forget gate? The following formula can be used to describe the forget gate:

[0115]

[0116] The output gate determines the output signal of the LSTM module at that moment. The formula is as follows:

[0117]

[0118] However, there is still a problem with the LSTM model: it cannot encode information from the back to the front. Therefore, it is necessary to use a bidirectional long short-term memory network (Bi-directional Long Short-Term Memory, Bi-LSTM) to capture bidirectional correlation information.

[0119] Furthermore, the fusing of the time domain features and the frequency domain features includes: fusing them in parallel.

[0120] Furthermore, extracting the fused features to obtain semantic features includes:

[0121] The Transformer network is used to extract the fused features to obtain semantic features.

[0122] The Transformer network model, which combines deep learning with traditional machine learning, reduces the impact of insufficient data on deep learning results to a certain extent. The Transformer network model consists of two parts: the Transformer network and the Support Vector Machine (SVM):

[0123] Transformer is a deep learning model architecture for natural language processing and other sequence-to-sequence tasks. It introduces a self-attention mechanism to avoid using recursion in neural networks and completely rely on the self-attention mechanism to draw the global dependency between input and output. The similarity calculation formula is as follows:

[0124]

[0125] Among them, Q, K, and V are obtained by linear transformation of the input word vector x, Q is the information to be queried, K is the queried vector, and V is the value obtained by the query; Transformer adopts an encoder-decoder architecture, which consists of 6 layers of encoders and decoders stacked separately. It is a model structure that avoids cycles. The input data passes through 6 layers of encoders and is output to the decoder of each layer to calculate attention. Each layer structure of the encoder contains two sub-layers: a multi-head attention layer and a feedforward connection layer; the decoder has three sub-layer structures, covering the multi-head attention layer, the multi-head attention layer, and the feedforward connection layer; each sub-layer is followed by a residual connection and a regularization layer. Since the calculation of Transformer abandons the recursion and convolution of the cyclic structure, it is impossible to simulate the position information of words in the text, so it needs to be artificially added through position encoding. Number the position of each word in the sentence, and each number corresponds to a vector. By combining the position vector and the word vector, a certain position information is introduced to each word. The formula is as follows:

[0126]

[0127] Among them, pos represents the position of the word in the sentence, d represents the dimension of the position encoding, 2i represents the even dimension, and 2i+1 represents the odd dimension (i.e., 2i≤d, 2i+1≤d).

[0128] Furthermore, the classification of semantic features to obtain depression recognition results includes:

[0129] Support vector machine was used to classify semantic features and obtain depression recognition results.

[0130] There are two types of support vector machine applications: linear and nonlinear. Its purpose is to cut all data in half on a hyperplane, so that all data can be divided into the smallest part. SVM performs outstandingly in solving problems such as "over-learning", "under-learning", and "local minima", and has many advantages in solving small sample, nonlinear and high-dimensional pattern recognition problems. Its basic idea is to map linearly inseparable samples one by one into multidimensional space through nonlinear mapping (kernel function), and find the optimal hyperplane so that linearly inseparable samples become classifiable with a low misclassification rate.

[0131] H is the classification hyperplane, H1 and H2 are parallel planes that are closest to the hyperplane H and pass through two types of sample points. The distance between H1 and H2 is the classification distance, and the points passed by the plane are support vectors, which are the optimal hyperplanes that maximize the classification distance. The support vector machine is defined as a function:

[0132] y=T(w·x)+b

[0133] Among them, w represents the high-dimensional mapping space, which is used to map the nonlinear features of x to the linear features of the high-dimensional space. w and b can be obtained by the penalty function:

[0134]

[0135] In the above formula, the first term is the sum of empirical risks, the second term is the penalty term, and γ is the penalty coefficient. γ can adjust the ratio between the two. It can be seen that as γ increases, the proportion of the sum of empirical risks relative to the penalty term increases. In order to increase the stability of the estimate and ensure the average distribution of the algorithm, ε-L(d, y) is used for empirical risk estimation. The formula is as follows:

[0136]

[0137] When the loss function value is 0, the function can predict completely correctly. When the loss function value is not 0, the function prediction has different degrees of error. When the linear classification becomes inseparable, add the slack variable ε to the formula i (ε i ≥0, i=1,2,3,…,n), we get:

[0138]

[0139] In the above formula, the first term reflects the expressive ability, and the second term reflects the empirical risk. The constraints of the above formula are as follows:

[0140]

[0141] The Lagrangian operators a and a * By introducing optimization constraints and Kuhn-Tucker conditions into regression analysis, we get the dual formula:

[0142]

[0143] The constraints of the above formula are:

[0144]

[0145] The Lagrangian factor satisfies a·a * =0, according to the Lagrange multiplier factor, the expected weight vector of the optimal hyperplane can be obtained as follows:

[0146]

[0147] The corresponding function discriminant is:

[0148]

[0149] In the formula, k(x,x i ) is the kernel function, k(x, xi )=T(x)T(x i ), that is, the value of the kernel function is equal to the vector x of the eigenvector T(x) and the eigenvector T(x i ) of the vector x i The present invention uses a Gaussian kernel function, which is defined as:

[0150]

[0151] Where: g is the width parameter of the Gaussian kernel function. The two important parameters, penalty coefficient γ and kernel function width g, directly affect the prediction accuracy of the model. Choosing a suitable combination of γ and g parameters is very important to ensure the classification ability of SVM.

[0152] Furthermore, the preprocessed heart sound signal is input into the trained depression state recognition model to obtain the recognition result of the depression state, wherein the training process includes: constructing a training set, the training set is the heart sound signal with known depression state recognition results; the depression state recognition results include: normal state, mild depression state, moderate depression state and severe depression state; inputting the training set into the depression state recognition model to train the model, and when the total loss function value of the model no longer decreases, stopping the training to obtain the trained depression state recognition model.

[0153] Among them, the total loss function is a weighted superposition of four loss functions, which comprehensively considers the classification accuracy, sample weight and prediction probability. The overall expression is as follows:

[0154] L(p t )=λ1L focal (p t )+λ2L corss-entropy (p t )

[0155] +λ3L MSE (p t ,y i )+λ4L smooth (p t )

[0156] Among them, p t is the model's predicted probability for the sample (for positive samples, p t is the probability of predicting a positive result; for negative samples, p t is the complement of the probability of prediction being negative); λ1, λ2, λ3, λ4 are the weights of each loss function, which are used to adjust their influence on the final goal, L focal (p t ) represents the Focal Loss function, L corss-entropy (p t) represents the cross entropy loss function, L MSE (p t ,y i ) represents the MSE loss function, L smooth (p t ) represents the Smooth Loss loss function.

[0157] Among them, Focal Loss: focuses on difficult samples and reduces the weight of simple samples. The Focal Loss loss function effectively solves the imbalance problem of positive and negative samples in classification tasks by introducing a dynamic scaling factor and a positive and negative sample weight adjustment mechanism, which can significantly improve the detection accuracy and recall rate of the model.

[0158] The Focal Loss loss function formula is as follows:

[0159] L focal (p t )=-α(1-p t ) γ log(p t )

[0160] Among them, p t is the model's predicted probability for the sample, α t It is a parameter for adjusting the weight of positive and negative samples, which is used to adjust the weight ratio between positive and negative samples; γ is a parameter for adjusting the weight of difficult and easy samples (also called focusing parameter).

[0161] Among them, the cross-entropy loss function (Cross-Entropy Loss) measures the degree of match between the predicted probability and the actual label. The cross-entropy loss function measures the difference between the model's predicted probability and the true distribution. The cross-entropy loss helps the model gradually adjust parameters and ultimately achieve the best classification effect.

[0162] The cross entropy loss function formula is as follows:

[0163]

[0164] Among them, p t is the model's predicted probability for the sample, y i is the true label of sample i. If the true category of sample i is the same as the predicted category, then y i =1 if the condition is positive, and 0 if it is negative; K represents the classification category (normal, mild, moderate and severe depression correspond to K=4).

[0165] Among them, Mean Squared Error (MSE): is used to measure the square error between the predicted probability and the actual label.

[0166] The MSE loss function is as follows:

[0167]

[0168] Among them, p t is the model's predicted probability for the sample, y i is the true label of sample i. If the true category of sample i is the same as the predicted category, then y i is 1, otherwise it is 0.

[0169] Among them, Smooth Loss: is used to control gradient smoothing and avoid violent parameter fluctuations.

[0170] Smooth Loss loss function, the formula is as follows:

[0171]

[0172] Among them, p t is the model's predicted probability for the sample, y i is the true label of sample i.

[0173] The Transformer network used in the present invention extracts the deep features of the heart sound signal, regularizes it, and then uses the principal component analysis (PCA) method to reduce the dimension of the pre-processed data, so as to achieve the purpose of reducing the amount of calculation required for optimizing the SVM performance on the basis of ensuring the recognition effect. The training set feature data obtained after processing is input into the SVM, and the SVM is used for training, and the particle swarm algorithm is used to optimize the γ and g parameters of the SVM, and find a suitable combination of γ and g parameters to improve the classification accuracy of the SVM, until the training is completed when the termination condition is met; the trained SVM is used to perform diagnostic classification on the test set feature data to obtain the diagnosis result.

[0174] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A depression state recognition system based on the combination of time and frequency domains, characterized by: include: An acquisition module, which is configured to: acquire a heart sound signal to be identified; A preprocessing module, which is configured to: preprocess the heart sound signal to be recognized, wherein the preprocessing includes: form conversion, downsampling and filtering processing; A depression state recognition module is configured to: input the preprocessed heart sound signal into the trained depression state recognition model to obtain a depression state recognition result; Among them, the trained depression state recognition model is used to extract the Mel-frequency cepstral coefficient MFCC features from the preprocessed heart sound signal, and extract the frequency domain features from the Mel-frequency cepstral coefficient MFCC features; at the same time, the time domain features are extracted from the preprocessed heart sound signal; after the time domain features and the frequency domain features are fused, the fused features are extracted to obtain semantic features; the semantic features are classified to obtain the depression recognition results.

2. The depression state recognition system based on the combination of time and frequency domains as claimed in claim 1, characterized in that: The step of extracting Mel frequency cepstral coefficient (MFCC) features from the preprocessed heart sound signal includes: The preprocessed heart sound signal is cropped; the cropped heart sound signal is framed; the frame processing result is windowed; the windowing processing result is Fourier transformed; based on the Fourier transform result, a Mel filter group is constructed; based on the Mel filter group, the logarithmic energy is calculated; the logarithmic energy is discrete cosine transformed to obtain the Mel frequency cepstral coefficient MFCC feature.

3. The depression state recognition system based on the combination of time and frequency domains as claimed in claim 2, characterized in that Calculating logarithmic energy according to the Mel filter group includes: calculating logarithmic energy according to the obtained Mel filter group, and the formula is as follows: Performing discrete cosine transform on the logarithmic energy to obtain the Mel frequency cepstrum coefficient MFCC feature, including: bringing the logarithmic energy into discrete cosine transform DCT, and finding the Mel frequency cepstrum coefficient MFCC coefficient, where the formula of discrete cosine transform DCT is as follows: Wherein, F(u) represents the Mel-frequency cepstral coefficient MFCC coefficient, N represents the number of triangular filters, when u=0, c(u)=N-0.5, otherwise c(u)=(N / 2)-0.

5.

4. The depression state recognition system based on the combination of time and frequency domains as claimed in claim 1, characterized in that: The extracting frequency domain features from the Mel-frequency cepstral coefficient MFCC features includes: A two-dimensional convolutional neural network is used to extract frequency domain features from the Mel-frequency cepstral coefficient MFCC features; The two-dimensional convolutional neural network comprises: a first two-dimensional convolutional layer, a second two-dimensional convolutional layer, a third two-dimensional convolutional layer, a fourth two-dimensional convolutional layer and a maximum pooling layer connected in sequence; The first two-dimensional convolution layer, the second two-dimensional convolution layer, the third two-dimensional convolution layer and the fourth two-dimensional convolution layer are used to extract features in the frequency domain. The number of filters in the four convolution layers increases layer by layer to ensure the hierarchical feature extraction process. The kernel size of all convolution layers is always maintained at (3, 3) to ensure local feature capture.

5. The depression state recognition system based on the combination of time and frequency domains as claimed in claim 1, characterized in that: The extracting time domain features from the preprocessed heart sound signal includes: extracting time domain features from the preprocessed heart sound signal using a Bi-LSTM neural network; The extracting the fused features to obtain the semantic features includes: using a Transformer network to extract the fused features to obtain the semantic features; The classifying of semantic features to obtain depression recognition results includes: using a support vector machine to classify semantic features to obtain depression recognition results.

6. The depression state recognition system based on the combination of time and frequency domains as claimed in claim 1, characterized in that: The preprocessed heart sound signal is input into the trained depression state recognition model to obtain the recognition result of the depression state, wherein the training process includes: constructing a training set, the training set is the heart sound signal with known depression state recognition result; the depression state recognition result includes: normal state, mild depression state, moderate depression state and severe depression state; inputting the training set into the depression state recognition model to train the model, and stopping the training when the total loss function value of the model no longer decreases to obtain the trained depression state recognition model.

7. The depression state recognition system based on the combination of time and frequency domains as claimed in claim 6, characterized in that: in, The total loss function is a weighted superposition of four loss functions, taking into account classification accuracy, sample weight and prediction probability. The overall expression is as follows: L(p t )=λ1L focal (p t )+λ2L corss-entropy (p t ) +λ3L MSE (p t ,y i )+λ4L smooth (p t ) Among them, p t is the predicted probability of the model for the sample; λ1, λ2, λ3, λ4 are the weights of each loss function, which are used to adjust their influence on the final goal, L focal (p t ) represents the Focal Loss function, L corss-entropy (p t ) represents the cross entropy loss function, L MSE (p t ,y i ) represents the MSE loss function, L smooth (p t ) represents the Smooth Loss loss function.

8. The depression state recognition system based on the combination of time and frequency domains as claimed in claim 7, characterized in that: Fo The cal Loss loss function formula is as follows: L focal (p t )=-α(1-p t ) γ log(p t ) Among them, p t is the model's predicted probability for the sample, α t It is a parameter for adjusting the weight of positive and negative samples, which is used to adjust the weight ratio between positive and negative samples; γ is a parameter for adjusting the weight of difficult and easy samples.

9. The depression state recognition system based on the combination of time and frequency domains as claimed in claim 7, characterized in that: The cross entropy loss function formula is as follows: Among them, p t is the model's predicted probability for the sample, y i is the true label of sample i. If the true category of sample i is the same as the predicted category, then y i is 1, otherwise it is 0; K represents the classification category; The MSE loss function is as follows: Among them, p t is the model's predicted probability for the sample, y i is the true label of sample i. If the true category of sample i is the same as the predicted category, then y i is 1 if true, 0 if false.

10. The depression state recognition system based on the combination of time and frequency domains as claimed in claim 7, characterized in that: SmoothLoss loss function, the formula is as follows: Among them, p t is the model's predicted probability for the sample, y i is the true label of sample i.

Images