Feature processing method based on voice recognition and terminal

Abstract

The invention relates to the technical field of machine learning, and discloses a depression feature processing method and terminal based on voice recognition, and the method comprises the steps: obtaining voice monitoring data, and extracting the information features of the voice monitoring data; inputting the information features into a time sequence feature extraction network, wherein the time sequence feature extraction network outputs time sequence features; and inputting the time sequence characteristics into an auxiliary information acquisition network, wherein the auxiliary information acquisition network outputs auxiliary information for assisting depression judgment. According to the invention, continuous voice monitoring is carried out on the patient, the voice monitoring data of the patient is obtained, the information characteristics of the voice monitoring data are extracted, and the time sequence analysis is carried out on the information characteristics of the voice monitoring data obtained for a long time to obtain the time sequence characteristics, so that whether the patient has depression or not can be judged in an auxiliary manner according to the time sequence characteristics; therefore, the burden of medical staff is reduced, and more depression patients can be diagnosed and treated in time in the early stage.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a feature processing method based on speech recognition, a terminal and a readable storage medium. Background Art

[0002] Depression is a mental illness that is harmful to human health. It is usually manifested by symptoms such as low mood and loss of interest. In severe cases, it is accompanied by cognitive dysfunction. The diagnosis of depression in the existing technology mainly relies on subjective assessments of scales and interviews, and is ultimately determined by clinical judgment of psychiatry.

[0003] However, due to lack of awareness, delayed diagnosis or economic constraints, patients often miss the best time for treatment. Therefore, early identification, diagnosis and treatment are crucial for the recovery of depression. However, since the information on depression requires the participation of professional medical personnel, but is limited by the number of professional medical personnel, many patients with depression fail to receive effective diagnosis in the early stages.

[0004] To this end, some reliable, user-friendly and fast automated feature processing methods are needed to provide reference for medical personnel, support medical professionals to identify depression in a timely manner, and continuously monitor the progress of treatment to reduce the burden on medical personnel and enable more patients with depression to be diagnosed and treated in a timely manner at an early stage. Summary of the invention

[0005] The purpose of the present invention is to provide a feature processing method based on speech recognition, aiming to solve the problem that the prior art diagnosis of depression lacks auxiliary tools and auxiliary information and relies on manual work.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0007] The present invention provides a feature processing method based on speech recognition, the feature processing method based on speech recognition comprising:

[0008] Acquire voice monitoring data, and extract information features of the voice monitoring data;

[0009] Inputting the information feature into a time series feature extraction network, and the time series feature extraction network outputs a time series feature;

[0010] The time series features are input into an auxiliary information acquisition network, and the auxiliary information acquisition network outputs auxiliary information for assisting depression judgment.

[0011] Furthermore, the extracting information features of the voice monitoring data specifically includes:

[0012] Performing a fast Fourier transform on the speech monitoring data to obtain a speech frequency domain signal;

[0013] Performing Mel filtering on the speech frequency domain signal to obtain a plurality of filtered signals;

[0014] Performing logarithmic processing on each of the filtered signals to obtain a plurality of logarithmic results;

[0015] Perform discrete cosine transform on each of the logarithmic results to obtain cepstral coefficients, and use the cepstral coefficients as the information features.

[0016] Furthermore, the temporal feature extraction network includes a long short-term memory network and a temporal convolutional network;

[0017] The inputting the information feature into a time series feature extraction network, wherein the time series feature extraction network outputs a time series feature, specifically includes:

[0018] Inputting the information features into the long short-term memory network to extract time relationship features;

[0019] Inputting the temporal relationship feature into the temporal convolutional network to obtain a temporal refinement feature;

[0020] The time relationship feature and the time refinement feature are used as the time series feature.

[0021] Furthermore, the step of inputting the information features into the long short-term memory network to extract the time relationship features specifically includes:

[0022] In each calculation, the information feature X of this calculation is t and the intermediate quantity H calculated last time t-1 Combined into a calculation vector [H t-1 ,X t ];

[0023] The vector [H t-1 ,X t ] are multiplied by the corresponding weights, and the product is added to the corresponding bias to obtain four intermediate vectors:

[0024] W o ·[H t-1 ,X t ]+b o ;

[0025] W g ·[H t-1 ,X t ]+b g ;

[0026] W i ·[H t-1 ,X t ]+b i ;

[0027] W f ·[H t-1 ,X t ]+b f ;

[0028] Among them, W o , W g , W i and W f are weights, b o , b g , b i and b f All are biased;

[0029] Input the four intermediate vectors into the activation function to obtain four intermediate parameters o t , g t 、i t and f t ;

[0030] According to the intermediate parameter o t , g t 、i t and f t , and the intermediate quantity C calculated last time t-1 Calculate the intermediate quantity of this calculation:

[0031] C t =f t ·C t-1 +g t ·i t ;

[0032] H t =o t HardTanh(C t );

[0033] Wherein, HardTanh represents the HardTanh function;

[0034] The intermediate quantity C t and H t Save and save the intermediate quantity H calculated this time t As a time relationship feature Output

[0035] The calculation formula of the HardTanh function is:

[0036]

[0037] Here, x represents the input variable.

[0038] Furthermore, the four intermediate vectors are respectively input into the activation function to obtain four intermediate parameters g t , o t 、f t and i t , specifically including:

[0039] The middle vector W g ·[H t-1 ,X t ]+b g and the intermediate vector W f ·[H t-1 ,X t ]+b f Input HardTanh function to get the intermediate parameter g t and f t :

[0040] g t =HardTanh(W g ·[H t-1 ,X t ]+b g );

[0041] f t =HardTanh(W f ·[H t-1 ,X t ]+b f );

[0042] The middle vector W o ·[H t-1 ,X t ]+b o and the intermediate vector W i ·[H t-1 ,X t ]+b i Input HardSigmoid function to get the intermediate parameter o t and i t :

[0043] o t =HardSigmoid(W o ·[H t-1 ,X t ]+b o );

[0044] i t =HardSigmoid(W i ·[H t-1 ,X t ]+b i );

[0045] Wherein, HardSigmoid represents the HardSigmoid function, and the calculation formula of the HardSigmoid function is:

[0046]

[0047] Furthermore, the auxiliary information acquisition network includes a cross attention module and a classification network, and the time series features are input into the auxiliary information acquisition network, and the auxiliary information acquisition network outputs auxiliary information for assisting depression judgment, specifically including:

[0048] Inputting the temporal features into the cross-attention module, the cross-attention module outputs key features;

[0049] The key features are input into the classification network, and the classification network outputs the auxiliary information.

[0050] Furthermore, the step of inputting the temporal features into the cross attention module, and the cross attention module outputting key features, specifically includes:

[0051] Separate the time refinement feature and the time relationship feature to obtain separate time refinement features and separate time relationship features;

[0052] A similarity matrix is ​​calculated based on the time-division feature and the time-division relationship feature;

[0053] Normalizing the similarity matrix according to the softmax function to obtain a weight matrix;

[0054] The key feature is calculated based on the weight matrix and the time-refined feature.

[0055] Furthermore, the classification network includes an average pooling layer and a fully connected layer;

[0056] The inputting the key feature into the classification network, and the classification network outputting the auxiliary information, specifically includes:

[0057] Inputting the key features into an average pooling layer, and the average pooling layer outputs a pooling result;

[0058] The pooling result is input into a fully connected layer, and the fully connected layer outputs the auxiliary information.

[0059] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, which includes: a memory, a processor, and a feature processing program based on speech recognition stored in the memory and executable on the processor, wherein the feature processing program based on speech recognition, when executed by the processor, controls the terminal to implement the steps of a feature processing method based on speech recognition as described above.

[0060] In addition, to achieve the above-mentioned purpose, the present invention also provides a readable storage medium, which stores a feature processing program based on speech recognition. When the feature processing program based on speech recognition is executed by a processor, the steps of a feature processing method based on speech recognition as described above are implemented.

[0061] The present invention adopts the above technical solution to achieve the following effects:

[0062] The present invention obtains the patient's voice monitoring data by performing continuous voice monitoring on the patient, extracts the information features of the voice monitoring data, and then performs time series analysis on the information features of the voice monitoring data obtained over a long period of time to obtain time series features, so as to assist in determining whether the patient has depression based on the time series features, thereby reducing the burden on medical personnel and enabling more patients with depression to receive early diagnosis and timely treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a flow chart of steps of a feature processing method based on speech recognition in a preferred embodiment of the present invention;

[0064] Figure 2 It is a schematic diagram of the principle of a feature processing method based on speech recognition in a preferred embodiment of the present invention;

[0065] Figure 3 is a schematic diagram of the structure of a first digital circuit in a preferred embodiment of the present invention;

[0066] Figure 4 is a schematic diagram of the structure of the second digital circuit in a preferred embodiment of the present invention;

[0067] Figure 5 It is a schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0069] Embodiment 1

[0070] See also Figure 1 and Figure 2 , Embodiment 1 of the present application is a feature processing method based on speech recognition, which comprises the steps of:

[0071] S1. Acquire speech monitoring data and extract information features of the speech monitoring data.

[0072] In traditional testing, depression can be identified through methods such as electroencephalogram (EEG), visual expression and speech analysis. Among them, EEG signals objectively reflect the internal state of the human brain and are considered to have broad prospects in promoting effective and efficient clinical diagnosis and identification of depression, as well as further improving clinical outcomes of treatment. However, EEG requires subjects to wear relevant collection and detection equipment in real time during the test, which reduces the convenience of monitoring and may cause inconvenience to patients with depression. This inconvenience may also affect the patient's mood and aggravate the patient's condition. In addition, due to the multi-channel and dense characteristics of EEG signals, it is challenging to optimize the spatial information of EEG, and the machine learning technology developed for depression identification faces major challenges.

[0073] Similarly, visual expressions require a high-level collection process and require the subjects to be constantly monitored under the camera, which greatly limits the monitoring environment and easily exposes the subjects' personal life privacy, leading to worsening of the patient's condition. Therefore, these two methods are not suitable for the development of auxiliary monitoring methods and equipment for depression.

[0074] Therefore, the present invention selects the patient's voice monitoring data as input. Voice can reflect a person's emotional state and emotional changes, and can be used to achieve real-time detection of patients' mental illnesses.

[0075] After acquiring the patient's voice monitoring data, this embodiment extracts features from the original audio signal through Mel frequency cepstral coefficients (MFCC). Mel frequency cepstral coefficients are a linear transformation of the logarithmic energy spectrum based on the nonlinear Mel scale of sound frequency. By utilizing the nonlinear perception characteristics of the human ear to sound frequency, the linear frequency is converted into the Mel frequency, and then cepstral analysis is performed in the Mel frequency domain to finally obtain the Mel frequency cepstral coefficients. This nonlinear representation method can well simulate the auditory system of the human ear, and therefore has a wide range of applications in the field of sound processing.

[0076] In this embodiment, the speech monitoring data is first preprocessed, and the high-frequency components in the speech monitoring data are highlighted and the non-stationarity of the speech monitoring data is reduced through pre-emphasis, overlapping framing and windowing processing.

[0077] Afterwards, the speech monitoring data is subjected to a fast Fourier transform (FFT) to convert the speech monitoring data from a time domain signal to a frequency domain signal to obtain a speech frequency domain signal. Then, a Mel filter is performed on the speech frequency domain signal. The Mel filter applies a series of triangular filters to simulate the auditory characteristics of the human ear. Each filter outputs a filtered signal. The logarithm of all filtered signals is taken to simulate the nonlinear perception of the human ear to the sound intensity to obtain a logarithmic result. The logarithmic result is subjected to a discrete cosine transform to extract the cepstral coefficients. The cepstral coefficients describe the envelope of the spectrum and are used as the characteristics of the speech monitoring data.

[0078] The features of the extracted speech monitoring data contain most of the effective information for speech recognition analysis, such as the speaker's voice features, speaking speed and emotions, which can be used for auxiliary analysis of depression.

[0079] S2. Input the information feature into a time series feature extraction network, and the time series feature extraction network outputs a time series feature.

[0080] In this embodiment, information features are analyzed by a neural network, which includes a time series feature extraction network and a classification network, wherein the time series feature extraction network includes a long short-term memory network (LSTM) and a time series convolutional network (TCN), the long short-term memory network is used to extract the time relationship features of the information features, and the time series convolutional network is used to refine the time relationship features to obtain time refined features, and the time relationship features and the time refined features are used as the time series features.

[0081] Please refer to Figure 3 The long short-term memory network is used to process the variability of the input length of the information feature and roughly extract the time relationship characteristics from the signal. Specifically, in this embodiment, the long short-term memory network is implemented by a first digital circuit structure, and the first digital circuit structure includes a control unit, a storage unit and a computing unit. The control unit is used to control the operation of the storage unit and the computing unit, the storage unit is used to temporarily store data, and the computing unit is used for calculation.

[0082] After the information feature is input into the long short-term memory network, it is first temporarily stored in the buffer of the storage unit. During the t-th calculation, the information feature X input into the long short-term memory network is t and the intermediate quantity H calculated last time t-1 and C t-1 , and the weights and biases enter the calculation unit, which calculates the information feature X t and the intermediate quantity H t-1 Calculate the information feature X t and the intermediate quantity H t-1 Merge into a vector [H t-1 ,X t ], vector [H t-1 ,Xt ] and the corresponding weight W o , W g , W i and W f After multiplication, the product and the corresponding bias b o , b g , b i and b f Add them together to get the processed four vectors W o ·[H t-1 ,X t ]+b o , W g ·[H t-1 ,X t ]+b g , W i ·[H t-1 ,X t ]+b i and W f ·[H t-1 ,X t ]+b f , and then the processed four vectors are processed by four activation functions respectively to obtain four intermediate parameters.

[0083] Specifically, considering that the traditional activation functions Sigmoid function and Tanh function are both nonlinear operators, and nonlinear operators are difficult to support on hardware, in this embodiment, the calculation of LSTM gating mechanism and state update is simplified, and HardTanh function and HardSigmoid function are used instead of Sigmoid function and Tanh function as activation functions, wherein the formulas of HardTanh function and HardSigmoid function are as follows:

[0084]

[0085] Among them, x represents the variable input to the activation function.

[0086] According to the above activation function, the intermediate parameters of the four long short-term memory networks are calculated:

[0087] g t =HardTanh(W g ·[H t-1 , X t ]+b g );

[0088] o t =HardSigmoid(W o ·[H t-1 ,X t ]+b o );

[0089] f t =HardTanh(W f ·[H t-1 ,X t ]+b f );

[0090] i t =HardSigmoid(W i ·[H t-1 ,X t ]+b i );

[0091] Then calculate the result based on the four intermediate parameters, that is, the new intermediate quantity H t and C t , the specific calculation formula is as follows:

[0092] C t =f t ·C t-1 +g t ·i t ;

[0093] H t =o t HardTanh(C t );

[0094] The new intermediate quantity H t and C t Stored in the storage unit for the next calculation, the new intermediate quantity H t It is also the temporal relationship feature of the output of the long short-term memory network

[0095] Afterwards, the output of the long short-term memory network is input into the sequential convolution network. It is worth noting that the present invention adopts a dual clock mechanism. Since the computational requirements of the long short-term memory network are significantly higher than those of other modules, it becomes the bottleneck of the entire system, causing the sequential convolution network to be idle while waiting for the results of the long short-term memory network. Therefore, a dual clock mechanism is introduced and resource allocation is optimized. The long short-term memory network works with a high-speed clock, while the sequential convolution network works with a slower clock. Thanks to the dual clock mechanism, the sequential convolution network module using a slower clock can effectively reduce power consumption by reducing the voltage conversion rate, thereby reducing circuit power consumption and improving circuit parallelism through reasonable resource allocation and adaptive switching circuits.

[0096] However, due to the different system clock frequencies of the LSTM network and the TSCN, it is necessary to transmit data to the TSCN securely and stably through a handshake signal, that is, to coordinate the LSTM network and the TSCN through the clock domain crossing (CDC) module, so as to transmit data to the TSCN securely and stably. The TSCN uses a one-dimensional convolution layer, and applies unidirectional convolution to the temporal relationship features to obtain temporal refinement features.

[0097] S3. Inputting the time series features into an auxiliary information acquisition network, and the auxiliary information acquisition network outputs auxiliary information for assisting depression judgment.

[0098] Specifically, in this embodiment, the auxiliary information acquisition network includes a cross-attention module and a classification network.

[0099] First, the cross-attention module is used to identify the key features in the temporal features. In this embodiment, the cross-attention module uses the temporal relationship features extracted by the long short-term memory network As query (Q), the temporal refinement features extracted by the temporal convolutional network A cross-attention method is adopted to identify key features as keys (K) and values ​​(V).

[0100] Specifically, we first use the time relationship feature As query (Q), the temporal refinement features extracted by the temporal convolutional network As the key (K), the similarity matrix is ​​obtained, and the attention matrix is ​​input into the softmax function to calculate the attention weight of each head. Since exponential calculation is required, the calculation cost is very high. For this reason, unlike the previous method of approximate calculation of the softmax function, in this embodiment, an optimized calculation method is adopted to speed up the calculation of the attention weight in the cross-attention module while maintaining the minimum calculation error. Based on the characteristic that each attention head in the cross-attention module only processes two features, an approximate calculation method is specially designed to directly provide the softmax function output, thereby reducing division operations and improving accuracy.

[0101] First, the query (Q), key (K) and value (V) need to be mapped through a linear layer (Linear), and then the heads are divided according to the number of attention heads to obtain the head-by-head time refinement features and head-by-head time relationship features. Then, the similarity matrix is ​​calculated based on the head-by-head time refinement features and head-by-head time relationship features, and then normalized by the softmax function:

[0102]

[0103] Among them, x1 is the first feature of the input softmax function, and x2 is the second feature of the input softmax function.

[0104] After normalization, a weight matrix is ​​obtained, and the product of the weight matrix and the split-time refinement feature as the value (V) is calculated, and the products are concatenated as the key feature.

[0105] Finally, the key features are input into the auxiliary information of the classification network, which includes an average pooling layer (Avgpooling) and a fully connected layer (Classification).

[0106] In this embodiment, the cross-attention module and the temporal convolutional network belong to the same digital circuit structure, namely, the second digital circuit structure. The second digital circuit structure also includes a control unit, a storage unit and a computing unit. The computing unit of the second digital circuit structure includes a multiplication matrix, an adder and an activation function. In the second digital circuit structure, the multiplication operation of the computing unit mainly depends on the reuse of the multiplication matrix, and the addition operation depends on the reuse of the adder. Even though this will take more time, this time is still much less than the calculation time of the long short-term memory network. Therefore, in practical applications, it will not cause a significant output slowdown.

[0107] Embodiment 2

[0108] See also Figure 5 Based on the above method, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 5 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0109] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed in the terminal, such as the program code of the installation terminal, etc. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a feature processing program 40 based on speech recognition is stored on the memory 20, and the feature processing program 40 based on speech recognition can be executed by the processor 10, thereby realizing the terminal in the present application.

[0110] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing related programs of the feature processing method based on speech recognition.

[0111] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0112] In one embodiment, when the processor 10 executes a feature processing program 40 based on speech recognition in the memory 20 , the steps of the feature processing method based on speech recognition as described above are implemented.

[0113] The memory 20 may be an internal storage unit of the terminal in some embodiments, such as a hard disk or memory of the terminal. The memory 20 may also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed in the terminal, such as the program code of the installation terminal, etc. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a feature processing program based on speech recognition is stored on the memory 20, and the feature processing program based on speech recognition can be executed by the processor 10, thereby realizing a feature processing method based on speech recognition in the present application.

[0114] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the feature processing method based on speech recognition.

[0115] Embodiment 3

[0116] This embodiment provides a storage medium, wherein the readable storage medium stores a feature processing program based on speech recognition, and when the feature processing program based on speech recognition is executed by a processor, the steps of the feature processing method based on speech recognition as described above are implemented.

[0117] In summary, the present invention obtains the patient's voice monitoring data through continuous voice monitoring of the patient, extracts the information features of the voice monitoring data, and then performs time series analysis on the information features of the voice monitoring data obtained over a long period of time to obtain time series features, so as to assist in determining whether the patient has depression based on the time series features, thereby reducing the burden on medical personnel and enabling more patients with depression to be diagnosed and treated in a timely manner at an early stage.

[0118] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0119] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, an optical disk, etc.

[0120] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A feature processing method based on speech recognition, characterized in that: The feature processing method based on speech recognition includes: Acquire voice monitoring data, and extract information features of the voice monitoring data; Inputting the information feature into a time series feature extraction network, and the time series feature extraction network outputs a time series feature; The time series features are input into an auxiliary information acquisition network, and the auxiliary information acquisition network outputs auxiliary information for assisting depression judgment.

2. A feature processing method based on speech recognition according to claim 1, characterized in that: The extracting of information features of the voice monitoring data specifically includes: Performing a fast Fourier transform on the speech monitoring data to obtain a speech frequency domain signal; Performing Mel filtering on the speech frequency domain signal to obtain a plurality of filtered signals; Performing logarithmic processing on each of the filtered signals to obtain a plurality of logarithmic results; Perform discrete cosine transform on each of the logarithmic results to obtain cepstral coefficients, and use the cepstral coefficients as the information features.

3. A feature processing method based on speech recognition according to claim 1, characterized in that: The temporal feature extraction network includes a long short-term memory network and a temporal convolutional network; The inputting the information feature into a time series feature extraction network, wherein the time series feature extraction network outputs a time series feature, specifically includes: Inputting the information features into the long short-term memory network to extract time relationship features; Inputting the temporal relationship feature into the temporal convolutional network to obtain a temporal refinement feature; The time relationship feature and the time refinement feature are used as the time series feature.

4. The method for processing depression features based on speech recognition according to claim 3, characterized in that: The step of inputting the information features into the long short-term memory network to extract the time relationship features specifically includes: In each calculation, the information feature X of this calculation is t and the intermediate quantity H calculated last time t-1 Combined into a calculation vector [H t-1 ,X t ]; The vector [H t-1 ,X t ] are multiplied by the corresponding weights, and the products are added to the corresponding biases to obtain four intermediate vectors: W o ·[H t-1 ,X t ]+b o ; W g ·[H t-1 ,X t ]+b g ; W i ·[H t-1 ,X t ]+b i ; W f ·[H t-1 ,X t ]+b f ; Among them, W o , W g , W i and W f are weights, b o 、b g 、b i and b f All are biased; Input the four intermediate vectors into the activation function to obtain four intermediate parameters o t , g t 、i t and f t ; According to the intermediate parameter o t , g t 、i t and f t , and the intermediate quantity C calculated last time t-1 Calculate the intermediate quantity of this calculation: C t =f t ·C t-1 +g t ·i t ; H t =o t ·HardTanh(C t ); Wherein, HardTanh represents the HardTanh function; The intermediate quantity C t and H t Save and save the intermediate quantity H calculated this time t As a time relationship feature Output The calculation formula of the HardTanh function is: Here, x represents the input variable.

5. A feature processing method based on speech recognition according to claim 4, characterized in that: The four intermediate vectors are respectively input into the activation function to obtain four intermediate parameters g t , o t 、f t and i t , including: The intermediate vector W g ·[H t-1 ,X t ]+b g and the intermediate vector W f ·[H t-1 ,X t ]+b f Input HardTanh function to get the intermediate parameter g t and f t : g t =HardTanh(W g ·[H t-1 ,X t ]+b g ); f t =HardTanh(W f ·[H t-1 ,X t ]+b f ); The intermediate vector W o ·[H t-1 ,X t ]+b o and the intermediate vector W i ·[H t-1 ,X t ]+b i Input HardSigmoid function to get the intermediate parameter o t and i t : o t =HardSigmoid(W o ·[H t-1 ,X t ]+b o ); i t =HardSigmoid(W i ·[H t-1 ,X t ]+b i ); Wherein, HardSigmoid represents the HardSigmoid function, and the calculation formula of the HardSigmoid function is:

6. A feature processing method based on speech recognition according to claim 3, characterized in that: The auxiliary information acquisition network includes a cross attention module and a classification network, the time series features are input into the auxiliary information acquisition network, and the auxiliary information acquisition network outputs auxiliary information for assisting depression judgment, specifically including: Inputting the temporal features into the cross-attention module, the cross-attention module outputs key features; The key features are input into the classification network, and the classification network outputs the auxiliary information.

7. A feature processing method based on speech recognition according to claim 6, characterized in that: The step of inputting the temporal features into the cross attention module, and the cross attention module outputting key features, specifically includes: Separate the time refinement feature and the time relationship feature to obtain separate time refinement features and separate time relationship features; A similarity matrix is ​​calculated based on the time-division feature and the time-division relationship feature; Normalizing the similarity matrix according to the softmax function to obtain a weight matrix; The key feature is calculated based on the weight matrix and the time-refined feature.

8. A feature processing method based on speech recognition according to claim 6, characterized in that: The classification network includes an average pooling layer and a fully connected layer; The inputting the key feature into the classification network, and the classification network outputting the auxiliary information, specifically includes: Inputting the key features into an average pooling layer, and the average pooling layer outputs a pooling result; The pooling result is input into a fully connected layer, and the fully connected layer outputs the auxiliary information.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a feature processing program based on speech recognition stored in the memory and executable on the processor. When the feature processing program based on speech recognition is executed by the processor, the terminal is controlled to implement the steps of a feature processing method based on speech recognition as described in any one of claims 1 to 8.

10. A readable storage medium, characterized in that: The readable storage medium stores a feature processing program based on speech recognition, and when the feature processing program based on speech recognition is executed by a processor, the steps of a feature processing method based on speech recognition as described in any one of claims 1 to 8 are implemented.

Images