Depression state recognition system based on multi-level feature fusion

Abstract

This invention discloses a depressive state recognition system based on multi-level feature fusion, comprising: inputting both cropped pulse signals and electrocardiogram (ECG) signals into a trained feature-level depressive state detection model for feature-level fusion and outputting feature-level depressive state classification results; inputting the cropped pulse signals into a trained pulse signal decision-level fusion subnetwork and outputting PPG-DFSN depressive state classification results; inputting the cropped ECG signals into a trained ECG signal decision-level fusion subnetwork and outputting ECG-DFSN depressive state classification results; performing decision-level fusion of the PPG-DFSN and ECG-DFSN depressive state classification results to obtain a decision-level fused depressive state classification result; and fusing the decision-level fused depressive state classification result with the feature-level depressive state classification result to obtain the final depressive state classification result.

Description

Technical Field

[0001] This invention relates to the field of depression state recognition technology, and in particular to a depression state recognition system based on multi-level feature fusion. Background Technology

[0002] The statements in this section merely refer to the background art related to this invention and do not necessarily constitute prior art.

[0003] Currently, the clinical diagnosis of depression is mainly based on the diagnostic criteria for depression in ICD-0 or DSM-V, combined with patient interviews, scales, and the doctor's clinical experience. Misdiagnosis is easily caused by subjective factors such as patient cooperation and doctor's skill level. Furthermore, the lack of patient awareness and early screening tools means that some patients are already diagnosed with major depressive disorder at the time of diagnosis.

[0004] In recent years, with the development of the internet and science and technology, deep learning has been widely applied in the medical field due to its powerful data processing and data mining capabilities. Utilizing machine learning and biological information to assist in depression identification has become an important way to improve the accuracy of depression identification. Traditional gated recurrent units (GRUs) often require a large amount of data for training, which is difficult to obtain in clinical practice. Furthermore, single physiological signals are difficult to reliably and accurately identify depression in deep learning. Currently, the application of biological signals mainly includes eye-tracking signals, facial expressions, speech signals, and electroencephalogram (EEG) signals. However, eye-tracking signals, facial expressions, and speech signals are deceptive, and EEG signal acquisition is difficult and expensive, making these unsuitable for widespread use in the population. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a depression state recognition system based on multi-level feature fusion. The data in this invention is readily available and inexpensive. The recognition model, built using a deep neural network, has the characteristic of providing results immediately upon inputting data, resulting in fast recognition speed.

[0006] A depressive state recognition system based on multi-level feature fusion includes:

[0007] The data acquisition module is configured to acquire the pulse signal and electrocardiogram signal to be identified.

[0008] The data preprocessing module is configured to preprocess the pulse signal and electrocardiogram signal to be identified.

[0009] The data trimming module is configured to trim the preprocessed signal.

[0010] The depression recognition module is configured to input both the cropped pulse signal and electrocardiogram signal into the trained feature-level depression state detection model. The trained feature-level depression state detection model performs feature-level fusion of pulse wave signal features and electrocardiogram signal features, and outputs feature-level depression state classification results.

[0011] The cropped pulse signal is input into the trained pulse signal decision-fusing subnetwork (PPG-DFSN) and the PPG-DFSN depression state classification result is output.

[0012] The cropped ECG signal is input into the trained Electrocardiogram-Decision Fusion Sub Network (ECG-DFSN) to output the ECG-DFSN depression state classification result.

[0013] The PPG-DFSN depression classification results and the ECG-DFSN depression classification results are fused at the decision level to obtain the decision-level fused depression classification results.

[0014] The final depression classification result is obtained by fusing the decision-level fusion depression classification result with the feature-level depression classification result.

[0015] One of the above technical solutions has the following advantages or beneficial effects:

[0016] By combining deep learning with traditional machine learning and using multi-level feature fusion processing, the impact of insufficient data on deep networks is reduced to some extent, resulting in more comprehensive feature extraction and higher recognition accuracy. In the clinical diagnosis of depression, compared with currently used scales, the data in this invention is readily available and inexpensive; the recognition model built using deep neural networks has the characteristic of providing results immediately upon inputting data, resulting in fast recognition speed; the method provided greatly reduces the subjective factors of doctors and patients, making the test results more objective. Attached Figure Description

[0017] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0018] Figure 1 This is a schematic diagram of the structure of the depression state recognition system based on GRU and multi-level feature fusion described in this invention.

[0019] Figure 2This is a flowchart illustrating the preprocessing of pulse and electrocardiogram signals according to an embodiment of the present invention;

[0020] Figure 3 This is a diagram showing the effect of filtering the pulse signal according to an embodiment of the present invention;

[0021] Figure 4 This is a diagram showing the effect of filtering the electrocardiogram signal according to an embodiment of the present invention;

[0022] Figure 5 This is a schematic diagram of the overall network architecture of the present invention;

[0023] Figure 6 This is a schematic diagram of the internal structure of a GRU network.

[0024] Figure 7 This is a schematic diagram of a support vector machine. Detailed Implementation

[0025] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0026] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. Furthermore, it should be understood that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0028] All data acquisition in this embodiment is carried out in accordance with laws and regulations and with user consent, and the data is used legally.

[0029] Example 1

[0030] This embodiment provides a depressive state recognition system based on multi-level feature fusion;

[0031] like Figure 1 As shown, the depressive state recognition system based on multi-level feature fusion includes:

[0032] The data acquisition module is configured to acquire the pulse signal and electrocardiogram signal to be identified.

[0033] The data preprocessing module is configured to preprocess the pulse signal and electrocardiogram signal to be identified.

[0034] The data trimming module is configured to trim the preprocessed signal.

[0035] The depression recognition module is configured to input both the cropped pulse signal and electrocardiogram signal into the trained feature-level depression state detection model. The trained feature-level depression state detection model performs feature-level fusion of pulse wave signal features and electrocardiogram signal features, and outputs feature-level depression state classification results.

[0036] The cropped pulse signal is input into the trained pulse signal decision-fusing subnetwork (PPG-DFSN) and the PPG-DFSN depression state classification result is output.

[0037] The cropped ECG signal is input into the trained Electrocardiogram-Decision Fusion Sub Network (ECG-DFSN) to output the ECG-DFSN depression state classification result.

[0038] The PPG-DFSN depression classification results and the ECG-DFSN depression classification results are fused at the decision level to obtain the decision-level fused depression classification results.

[0039] The final depression classification result is obtained by fusing the decision-level fusion depression classification result with the feature-level depression classification result.

[0040] Furthermore, acquiring the pulse signal and electrocardiogram signal to be identified includes:

[0041] Acquire the pulse signal and electrocardiogram signal to be identified;

[0042] The pulse and electrocardiogram signals to be identified are converted into one-dimensional matrices.

[0043] It should be understood that acquiring the pulse and electrocardiogram (ECG) signals to be identified refers to the pulse and ECG signals of the test subject in a resting state. Physiological signal acquisition equipment is used to collect the resting pulse and ECG signals of the test subject. During signal acquisition, it is essential to ensure that the test subject is in a relatively enclosed environment, while avoiding interference from noise, electromagnetic signals, and other factors. The RM6280C multi-channel physiological data acquisition and processing system is used to acquire the pulse and ECG signals of the test subject. The test subject lies supine on the experimental table, a finger cot for measuring pulse signals is attached to the right index finger of the test subject, and electrode clips for measuring ECG signals are clamped above the right wrist and left and right ankle joints of the test subject. After preparation, remind the test subject to close their eyes and relax their whole body during the data collection process. Observe the pulse and ECG signals in the multi-channel physiological signal acquisition and processing system software. If the recording requirements are met, click the record / end button to start collecting pulse and ECG signals. If abnormal signals occur during the process, the collection time can be appropriately extended to ensure that the duration of continuous and effective signals is collected. Save and export the collected signals according to their sequence numbers.

[0044] Specifically, clinically, depression is often accompanied by parasympathetic neurasthenia and sympathetic hyperactivity, leading to problems such as increased heart rate, increased cardiac contractility, and elevated arterial blood pressure. To effectively reflect these physiological changes, this invention selects pulse signals and electrocardiogram signals and performs multi-level feature fusion to identify depressive states. This method is simple, easy to implement, and highly practical.

[0045] Furthermore, such as Figure 2 As shown, the preprocessing of the pulse signal and electrocardiogram signal to be identified includes:

[0046] The pulse and electrocardiogram signals of the one-dimensional matrix are downsampled;

[0047] The downsampled signal is processed using a smoothing filter algorithm to remove power frequency interference.

[0048] After removing power frequency interference, the signal is processed using a median filtering algorithm to remove electromyographic noise.

[0049] The signal after removing electromyographic noise was processed using a wavelet transform algorithm to remove baseline drift.

[0050] Figure 3 This is a diagram showing the effect of filtering the pulse signal according to an embodiment of the present invention; Figure 4 This is a diagram showing the effect of filtering the electrocardiogram signal according to an embodiment of the present invention.

[0051] Furthermore, the trimming of the preprocessed signal includes:

[0052] The signal is trimmed to a set length using a sliding window method.

[0053] Furthermore, the process of using wavelet transform algorithm to remove baseline drift specifically includes:

[0054] Wavelet transform algorithm is used to denoise the pulse signal and ECG signal. The sym8 wavelet basis is selected to decompose and reconstruct the pulse signal and ECG signal to remove the baseline drift of the pulse signal and ECG signal.

[0055] It should be understood that median filtering and smoothing filtering are used to denoise the signal.

[0056] Median filtering is a commonly used image processing algorithm, but it can also be used for filtering one-dimensional data. Median filtering is a non-linear signal processing technique based on order statistics theory that can effectively suppress noise. The basic principle of median filtering is to replace the value of a point in a digital sequence with the median value of all points in its neighborhood, making the surrounding data values ​​closer to the true value, thereby eliminating isolated noise points.

[0057] Data smoothing filtering is a signal processing technique used to reduce noise and fluctuations in data, allowing for better extraction of signal trends and characteristics. Its principle is based on using a filter to perform a weighted average of the input signal to suppress high-frequency noise components, thus achieving a smoothing effect. A fixed-size window is used to calculate the average value of the data within the window, and this average value is used as the output. The window can be a simple rectangular window or a window with weighting factors. The size of the filter's window determines the degree of smoothing.

[0058] Furthermore, such as Figure 5 As shown, the cropped pulse signal and electrocardiogram signal are both input into the trained feature-level depressive state detection model. The network structure of the trained feature-level depressive state detection model includes:

[0059] The first and second GRU networks are parallel;

[0060] The input terminal of the first GRU network is used to input the cropped pulse signal;

[0061] The output of the first GRU network is connected to the input of the feature fusion module;

[0062] The input terminal of the second GRU network is used to input the cropped electrocardiogram signal;

[0063] The output of the second GRU network is connected to the input of the feature fusion module;

[0064] The feature fusion module is used to concatenate and splice the input pulse signal features and electrocardiogram signal features to obtain fused features; the output of the feature fusion module is connected to the input of the first support vector machine classifier.

[0065] The first support vector machine classifier classifies the fused features to obtain the classification result of the first depressive state.

[0066] Furthermore, the cropped pulse signal and electrocardiogram signal are both input into the trained feature-level depressive state detection model, wherein the training process of the trained feature-level depressive state detection model includes:

[0067] A training set is constructed, which consists of pulse signals and electrocardiogram signals with known depressive state detection results; the depressive state detection results include both normal and depressive results;

[0068] The training set is input into the feature-level depression detection model to train the model. Training is stopped when the loss function value of the model no longer decreases or the number of iterations exceeds the set number, and the trained feature-level depression detection model is obtained.

[0069] It should be understood that the training set includes multiple segments of pulse signals and electrocardiogram signals, as well as labels indicating whether each pulse signal and electrocardiogram signal segment is normal or depressed.

[0070] Furthermore, the cropped pulse signal is input into the trained pulse signal decision-level fusion sub-network to output the PPG-DFSN depression state classification result. The trained pulse signal decision-level fusion sub-network includes a third GRU network and a second support vector machine (SVM) classifier connected in sequence.

[0071] Furthermore, the cropped pulse signal is input into the trained pulse signal decision-level fusion subnetwork, and the PPG-DFSN depression state classification result is output. The training process of the trained pulse signal decision-level fusion subnetwork includes:

[0072] The training set is input into the pulse signal decision-level fusion subnetwork to train the model. Training is stopped when the loss function value of the model no longer decreases or the number of iterations exceeds the set number, and the trained pulse signal decision-level fusion subnetwork is obtained.

[0073] Furthermore, the cropped ECG signal is input into the trained ECG signal decision-level fusion subnetwork to output the ECG-DFSN depression state classification result. The trained ECG signal decision-level fusion subnetwork includes a fourth GRU network and a third support vector machine classifier connected in sequence.

[0074] Furthermore, the cropped ECG signal is input into the trained ECG signal decision-level fusion subnetwork, and the ECG-DFSN depression state classification result is output. The training process of the trained ECG signal decision-level fusion subnetwork includes:

[0075] The training set is input into the ECG signal decision-level fusion subnetwork to train the model. Training is stopped when the loss function value of the model no longer decreases or the number of iterations exceeds the set number, and the trained ECG signal decision-level fusion subnetwork is obtained.

[0076] Further, the step of performing decision-level fusion of the PPG-DFSN depression classification results and the ECG-DFSN depression classification results to obtain a decision-level fused depression classification result includes:

[0077] Based on the Dezert-Smarandache Theory (Dezert-Smarandache-Thoery, DSmT) inference method, the PPG-DFSN depression classification results and the ECG-DFSN depression classification results are fused at the decision level to obtain the decision-level fused depression classification results.

[0078] Furthermore, the fusion method based on DSmT inference, which performs decision-level fusion of the PPG-DFSN depression classification results and the ECG-DFSN depression classification results to obtain a decision-level fused depression classification result, specifically includes:

[0079] There are N scene categories, and D N Θ ={θ2,...,θ N}, where {θ1, ..., θ} N} represent N different scene categories, where N is 2, and θ1 represents the feature-level depression state classification result V. F θ2 represents the decision-level fusion classification result of the depressive state V. D .

[0080] The DSmT classic combinatorial rules are used for combinatorial processing, and the proportional conflict redistribution rules (PCR5) are used to allocate the conflicting focal mass generated during the fusion process.

[0081] The PCR5 allocation rule assigns conflict quality (i.e. conflict focal mass) to each element involved in the conflict, and it is currently the most accurate conflict allocation rule.

[0082] For two sources of evidence S1 and S2 under the same identification framework Θ, namely V as described in this invention F and V D Their basic reliability assignment functions are m1(·) and m2(·), respectively, for any There are combinatorial rules based on PCR5:

[0083]

[0084] in

[0085]

[0086] m obtained after fusion PCR5 (·) can be used as a generalized basic reliability assignment. As the decision-level fusion result described in this invention, it still satisfies m PCR5 (·):D Θ →[0,1].

[0087] Furthermore, the process of fusing the decision-level fusion depression state classification result with the feature-level depression state classification result to obtain the final depression state classification result includes:

[0088] When the characteristic level of the depressive state classification result V F The fusion of decision-level depression classification results V D The largest element x in j When (j = 1, 2, ..., N) is greater than the set threshold T, take x. j The corresponding category is taken as the final classification result C;

[0089] When x j When the threshold T is less than or equal to the set threshold, feature-level fusion and decision-level fusion are used as two sources of evidence, and the feature-level depression state classification result V is used. F The fusion of decision-level depression classification results V D Each of them was assigned a basic reliability score, and then the results were fused using the DSmT method to obtain the final result V. FD Take V FD The category corresponding to the largest element in the classification is taken as the final classification result C.

[0090] The model employed integrates traditional machine learning with deep learning. A preprocessing module downsamples and denoises the one-dimensional matrix-form pulse and ECG signals, then trims them into signal segments of a predetermined length. The processed pulse and ECG signals are then fed into a GRU network to extract features. These features are used for feature-level fusion and decision-level fusion, respectively, and the results of feature-level and decision-level fusion are adaptively refused. Finally, the classification result is output based on the refusion result.

[0091] It should be understood that the feature-level module, PPG-DFSN, ECG-DFSN, and decision-level fusion module GRU network all share a core "update gate." While maintaining the memory capacity of neurons, it simplifies the structure of similar Long Short-Term Memory (LSTM) networks, thus improving training speed. The update gate determines how much previous information the model should pass to the next stage. Its reset gate is also crucial, determining how much previous information should be discarded. The next step activates the current memory content, and the activated update gate controls the amount of new and old information retained. New and old information are complementary. If a large amount of new information is retained, less old information will be considered, and vice versa. The specific structure is as follows... Figure 6 As shown.

[0092] Update gate formula:

[0093] z t =σ(W (z) x t +U (z) h t-1 )

[0094] Reset door formula:

[0095] r t =σ(W (r) x t +U (r) h t-1 )

[0096] Current memory content:

[0097] h′ t =tanh(Wx t +r t ⊙Uh t-1 )

[0098] Final memory of the current time step:

[0099] h t =z t ⊙h t-1 +(1-z t )⊙h′ t

[0100] In the formula: W (z) W (r) W and H represent the weights of the update gate, reset gate, and current memory, respectively; h t-1 h t These are the outputs from the previous time step and the current time step, respectively; x t This is the input for the current moment; U (z) U (r)U represents the weights of the circular connection; σ and tanh represent the sigmoid function and the tanh function, respectively; ☉ represents element-wise multiplication.

[0101] Support Vector Machine (SVM) is a quadratic classification method. With modifications, it can also be applied to multi-class classification problems. SVM applications come in two types: linear and non-linear. Its goal is to divide all data into two halves on a hyperplane, thus separating all data into the smallest possible subsets. SVM excels in solving problems such as overlearning, underlearning, and local minima, and demonstrates significant advantages in handling small sample sizes, non-linear problems, and high-dimensional pattern recognition. Its basic idea is to map linearly inseparable samples one-to-one into a multi-dimensional space through a non-linear mapping (kernel function), searching for the optimal hyperplane that makes the linearly inseparable samples classifiable with a low misclassification rate.

[0102] Figure 7 This is a schematic diagram of the SVM principle. Circles and crosses represent two different types of samples, respectively. H is the classification hyperplane, and H1 and H2 are parallel planes parallel to H that pass through points of the two sample types and are closest to the hyperplane H. The distance between H1 and H2 is the classification distance, and the points passed through by the planes are support vectors. The support vectors are the optimal hyperplane that maximizes the classification distance. The Support Vector Machine is defined as a function:

[0103] y = T(w·x) + b

[0104] Here, w represents the high-dimensional mapping space, which maps the nonlinear characteristics of x to linear characteristics in the high-dimensional space. w and b can be obtained from the penalty function:

[0105]

[0106] In the above formula, the first term is the sum of empirical risks, the PPG-DFSN term is the penalty term, and γ is the penalty coefficient. γ can adjust the ratio between the two; it can be seen that as γ increases, the proportion of the sum of empirical risks relative to the penalty term increases. To increase the stability of the estimation and ensure the average distribution of the algorithm, ε-L(d, y) is used for empirical risk prediction.

[0107]

[0108] When the loss function value is 0, the function can predict perfectly correctly; when the loss function value is not 0, the function prediction has varying degrees of error. When linear classification becomes inseparable, a slack variable ε is added to the equation. i (ε i ≥0, i=1,2,3,…,n), we get

[0109]

[0110] In the above formula, the first term reflects expressive ability, and the second term reflects experiential risk. The constraints of the above formula are as follows:

[0111]

[0112] Let the Lagrange operators a and a * By introducing optimization constraints and incorporating the Kuhn-Tucker condition into the regression analysis, we obtain the dual formula:

[0113]

[0114] The constraints of the above equation are:

[0115]

[0116] The Lagrange factor satisfies a·a * =0, and based on the Lagrange multiplier factor, the expected weight vector of the optimal hyperplane can be obtained as follows:

[0117]

[0118] The corresponding discriminant is:

[0119]

[0120] In the formula, k(x, x) i ) is the kernel function, k(x, x) i )=T(x)T(x i That is, the value of the kernel function is equal to the vector x of the eigenvector T(x) and the eigenvector T(x). i The vector x) i The inner product of the two. This invention uses a Gaussian kernel function, defined as:

[0121]

[0122] In the formula, g is the width parameter of the Gaussian kernel function. The penalty coefficient γ and the kernel function width g are two important parameters that directly affect the prediction accuracy of the model. Choosing an appropriate combination of γ and g parameters is crucial to ensuring the classification ability of SVM.

[0123] This invention extracts deep features from the feature set network, PPG-DFSN, ECG-DFSN, and the decision-level fusion module GRU network, regularizes them, and then uses principal component analysis (PCA) to reduce the dimensionality of the preprocessed data. This aims to reduce the computational cost required for optimizing SVM performance while maintaining recognition accuracy. The processed training set feature data is then input into the SVM for training. The particle swarm optimization algorithm is used to optimize the γ and g parameters of the SVM, finding a suitable combination of γ and g parameters to improve the classification accuracy of the SVM. Training is completed when a termination condition is met. The trained SVM is then used to perform diagnostic classification on the test set feature data to obtain the diagnostic results.

[0124] This invention utilizes complementary information between different fusion levels, and selects fusion methods based on serial connection and DSmT inference at the feature level and decision level, respectively. Some specific contents and processes are described below.

[0125] DSmT is an extension of the traditional DST, primarily used to handle the fusion of uncertain, highly conflicting, and imprecise information sources. In pattern recognition, DSmT theory is often used for decision-level fusion of multiple classifier outputs. The biggest difference between DSmT and DST theory is the inclusion of conflict information in the identification frame. Let Θ = {θ1, θ2} be an identification frame. In the power set 2... Θ Basic confidence values ​​are assigned on the set D = {θ1, θ2, θ1∪θ2}, while DSmT assigns confidence values ​​on the superpower set D. Θ Basic confidence assignment is performed on the superpower set space = {θ1, θ2, θ1∪θ2, θ1∩θ2}. Due to the existence of intersection operations in the superpower set, contradictory focal elements are preserved in the identification framework. This example uses the DSmT fusion method for decision-level fusion and re-fusion, assigning confidence values ​​only to the single focal elements in the superpower set space, i.e., for the superpower set space D... Θ Only θ1 and θ2 are assigned confidence values, while θ1∪θ2 and θ1∩θ2 are assigned zero confidence values.

[0126] The specific implementation process of the multi-level feature fusion-based depression state recognition system is as follows:

[0127] (1) First, the training samples are fed into the GRU network to extract deep features, resulting in deep feature vectors f1 and f2 of the pulse signal and electrocardiogram signal, as well as the feature vector f after feature-level fusion. c Set f1, f2, and f c The samples are fed into an SVM classifier to obtain SVM1, SVM2, and SVM3 respectively. c .

[0128] (2) For the patient’s data to be identified, the data is first sent to the GRU network to extract deep features, and then the features are fused at the feature level and at the decision level respectively.

[0129] (3) During the feature-level fusion process, the two vectors are concatenated end to end to form the feature-level fusion vector f. c , will f c Feed into the pre-trained SVM c The classification and identification process is performed to obtain the classification result V. F ;

[0130] In the decision-level fusion process, the feature vectors f1 and f2 extracted by the GRU network are fed into the corresponding classifiers SVM1 and SVM2 for classification and recognition, respectively, and the results are j1 and j2. The classifiers are then fused using the PCR5-based combination rule to obtain the fusion result V. D .

[0131] (4) During the re-fusion process, based on the feature-level fusion output V F Decision-level fusion output V D Perform adaptive fusion.

[0132] When V F and V D The largest element x in j When (j = 1, 2, ..., N) is greater than the set threshold T, take x. j The corresponding category is taken as the final classification result C; when x j When T is less than or equal to T, the feature-level fusion submodule and the decision-level fusion submodule are used as two sources of evidence, and V is used as the basis for the evidence. F and V D Each of them was assigned a basic reliability score, and then the results were fused using the DSmT method to obtain the final result V. FD Take V FD The category corresponding to the largest element in the classification is taken as the final classification result C.

[0133] (5) Output C as the final result, and the recognition ends.

[0134] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A depression state recognition system based on multi-level feature fusion, characterized in that, The method comprises the following steps: a data acquisition module configured to acquire a pulse signal and an electrocardiogram signal to be identified; a data preprocessing module configured to preprocess the pulse signal and the electrocardiogram signal to be identified; the preprocessing of the pulse signal and the electrocardiogram signal to be identified comprises: down-sampling the pulse signal and the electrocardiogram signal of the one-dimensional matrix; processing the down-sampled signal using a smoothing filter algorithm to remove power frequency interference; processing the signal after removing the power frequency interference using a median filter algorithm to remove electromyographic noise; processing the signal after removing the electromyographic noise using a wavelet transform algorithm to remove baseline drift; the processing using the wavelet transform algorithm to remove the baseline drift specifically comprises: using the wavelet transform algorithm to denoise the pulse signal and the electrocardiogram signal, selecting a sym8 wavelet basis to decompose and reconstruct the pulse signal and the electrocardiogram signal to remove the baseline drift of the pulse signal and the electrocardiogram signal; a data cropping module configured to crop the preprocessed signal; a depression recognition module configured to input the cropped pulse signal and the electrocardiogram signal into a trained feature-level depression state detection model, the trained feature-level depression state detection model performing feature-level fusion on pulse wave signal features and electrocardiogram signal features and outputting a feature-level depression state classification result; inputting the cropped pulse signal into a trained pulse signal decision-level fusion subnetwork to output a PPG-DFSN depression state classification result; inputting the cropped electrocardiogram signal into a trained electrocardiogram signal decision-level fusion subnetwork to output an ECG-DFSN depression state classification result; performing decision-level fusion on the PPG-DFSN depression state classification result and the ECG-DFSN depression state classification result to obtain a decision-level fusion depression state classification result; performing fusion on the decision-level fusion depression state classification result and the feature-level depression state classification result to obtain a final depression state classification result.

2. The depression state recognition system based on multi-level feature fusion according to claim 1, characterized in that, the acquisition of the pulse signal and the electrocardiogram signal to be identified comprises: acquiring the pulse signal and the electrocardiogram signal to be identified; converting the pulse signal and the electrocardiogram signal to be identified into a one-dimensional matrix.

3. The depression state recognition system based on multi-level feature fusion according to claim 1, characterized in that, the inputting of the cropped pulse signal and the electrocardiogram signal into the trained feature-level depression state detection model, wherein the network structure of the trained feature-level depression state detection model comprises: a first GRU network and a second GRU network arranged in parallel; the input end of the first GRU network is used for inputting the cropped pulse signal; the output end of the first GRU network is connected with the input end of a feature fusion module; the input end of the second GRU network is used for inputting the cropped electrocardiogram signal; the output end of the second GRU network is connected with the input end of the feature fusion module; the feature fusion module is used for concatenating and splicing the input pulse signal features and electrocardiogram signal features to obtain fusion features; the output end of the feature fusion module is connected with the input end of a first support vector machine classifier; the first support vector machine classifier performs classification processing on the fusion features to obtain a first depression state classification result.

4. The depression state recognition system based on multi-level feature fusion according to claim 1, characterized in that, The cropped pulse signal and the electrocardiogram signal are input into the trained feature-level depression state detection model, wherein the training process of the trained feature-level depression state detection model comprises: A training set is constructed, and the training set is a pulse signal and an electrocardiogram signal with known depression state detection results; the depression state detection results include two results of normal and depression; The training set is input into the feature-level depression state detection model to train the model, and when the loss function value of the model no longer decreases or the iteration number exceeds the set number of times, the training is stopped to obtain the trained feature-level depression state detection model.

5. The depression state recognition system based on multi-level feature fusion according to claim 1, characterized in that, The cropped pulse signal is input into the trained pulse signal decision-level fusion sub-network to output a PPG-DFSN depression state classification result, wherein the trained pulse signal decision-level fusion sub-network comprises a third GRU network and a second support vector machine classifier connected in sequence; The cropped pulse signal is input into the trained pulse signal decision-level fusion sub-network to output a PPG-DFSN depression state classification result, wherein the training process of the trained pulse signal decision-level fusion sub-network comprises: The training set is input into the pulse signal decision-level fusion sub-network to train the model, and when the loss function value of the model no longer decreases or the iteration number exceeds the set number of times, the training is stopped to obtain the trained pulse signal decision-level fusion sub-network.

6. The depression state recognition system based on multi-level feature fusion according to claim 1, characterized in that, The cropped electrocardiogram signal is input into the trained electrocardiogram signal decision-level fusion sub-network to output an ECG-DFSN depression state classification result, wherein the trained electrocardiogram signal decision-level fusion sub-network comprises a fourth GRU network and a third support vector machine classifier connected in sequence; The cropped electrocardiogram signal is input into the trained electrocardiogram signal decision-level fusion sub-network to output an ECG-DFSN depression state classification result, wherein the training process of the trained electrocardiogram signal decision-level fusion sub-network comprises: The training set is input into the electrocardiogram signal decision-level fusion sub-network to train the model, and when the loss function value of the model no longer decreases or the iteration number exceeds the set number of times, the training is stopped to obtain the trained electrocardiogram signal decision-level fusion sub-network.

7. The depression state recognition system based on multi-level feature fusion according to claim 1, characterized in that, The PPG-DFSN depression state classification result and the ECG-DFSN depression state classification result are decision-level fused to obtain a decision-level fusion depression state classification result, comprising: The PPG-DFSN depression state classification result and the ECG-DFSN depression state classification result are decision-level fused based on a Dezert-Smarandache theory reasoning fusion mode to obtain a decision-level fusion depression state classification result.

8. The depression state recognition system based on multi-level feature fusion according to claim 1, characterized in that, The decision-level fusion depression state classification result and the feature-level depression state classification result are fused to obtain a final depression state classification result, comprising: When the feature-level depression status classification result V F and the decision-level fused depression status classification result V D is the maximum element x j greater than a set threshold T , take x j the corresponding class as the final classification result C ; j = 1, 2, …, N ; When x j less than or equal to a set threshold value T , the feature level fusion and the decision level fusion are taken as two evidence sources, and the feature level depression state classification result V F and the decision level fusion depression state classification result V D are respectively given a basic confidence value, and then the results are fused by using a DSmT mode to obtain a result V FD , the class corresponding to the maximum element in V FD is taken as a final classification result C .

9. The depression state recognition system based on multi-level feature fusion according to claim 1, characterized in that, The system specific implementation process is as follows: (1) Firstly, the training samples are sent into the GRU network to extract deep features, obtaining the deep feature vectors of the pulse signal and the electrocardiogram signal f 1 and f 2, and the feature vectors after feature-level fusion of them f c ; the f 1, f 2, and f c are respectively sent into the SVM classifier, obtaining SVM1, SVM2, and SVM c ; (2) For the patient's to-be-recognized data, first input into the GRU network to extract deep features, and then the features are respectively subjected to feature-level fusion and decision-level fusion; (3) In the feature level fusion process, the two vectors are connected head to tail to form a feature level fusion vector f c , the f c is sent into the pre-trained SVM c for classification and recognition, and a classification result is obtained V F ; In the decision level fusion process, the feature vectors extracted by the GRU network f 1, f 2 are sent into the corresponding classifiers SVM1 and SVM2 for classification recognition, and the results are j 1 and j 2. The combination rule based on PCR5 is used to make DSmT decision level fusion of the classifiers, and the fusion result is V D ; (4) In the re-fusion process, according to the feature-level fusion output V F and the decision-level fusion output V D Adaptive fusion is performed: when V F and V D the maximum element in x j is greater than a set threshold T , take x j the corresponding category as the final classification result C ; When x j Less than or equal to T , the feature level fusion sub-module and the decision level fusion sub-module are taken as two evidence sources, and the DS mT method is used to fuse them to obtain the result V F and V D The basic confidence values are respectively assigned to them, and the DS mT method is used to fuse them to obtain the result V FD , and the class corresponding to the maximum element in V FD is taken as the final classification result C ; (5) output C As a final result, the recognition ends.

Images