Man-machine fusion-based depression auxiliary identification method and equipment

Abstract

The invention discloses a depression auxiliary identification method and device based on man-machine fusion, and the method comprises the steps: obtaining an EEG signal, carrying out the data preprocessing of the EEG signal, and carrying out the LZ complexity analysis; then, analyzing a signal by using a CNN-LSTM confidence analysis model, performing model evaluation judgment and result verification by using improved k-fold cross validation, and feeding back the confidence obtained by feedback, the original electroencephalogram and the LZ complexity to an expert for diagnosis and decision making; and finally, determining a TMS stimulation target by combining deep learning and a neural navigation system, determining stimulation parameters by using IAF, performing TMS stimulation, retaining stimulation positions and parameters, and displaying the stimulation positions and parameters to experts for further decision making and judgment. The LZ complexity can be used as an auxiliary algorithm to distinguish normal people from depression patients. The method focuses on the LZ complexity, and the LZ complexity can also be used as the input of CNN-LSTM, so that front and back connection and integration are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of human-machine fusion, and in particular to a method and device for auxiliary identification of depression based on human-machine fusion. Background Art

[0002] According to a survey by the World Health Organization, about 1 billion people in the world suffer from mental disorders. Depression is the most common mental illness in the world, but less than 50% of patients receive effective treatment.

[0003] There is a lack of objective diagnostic criteria for depression identification and diagnosis, and it is difficult to make a breakthrough. In the past, depression was generally identified and diagnosed by doctors with rich clinical experience according to the two diagnostic criteria of the Diagnostic and Statistical Manual of Mental Disorders (DSM) and the International Classification of Diseases (ICD). In this process, the doctor's subjective decision is the main diagnosis criterion for depression. Such a diagnostic process lacks the support of objective evidence and requires very high clinical experience from doctors, which inevitably leads to errors in medical diagnosis, missed diagnosis or misdiagnosis, which often occurs, affecting the subsequent treatment and prognosis of individuals suffering from depression. How to quickly and accurately realize the diagnosis and treatment of depression is a difficult problem that needs to be solved in the field of depression.

[0004] The normal brain is active all the time, which leads to complex and diverse EEG signals. The complexity between brain signals can be used to describe different brain states and is a key measure of system complexity. EEG signals are non-stationary nonlinear signals. Many researchers have chosen nonlinear analysis methods to study depression, such as neural networks and nearest neighbor classifiers, but have not used the complexity between EEG signals as the input of CNN-LSTM to study depression differentiation.

[0005] There are some defects in existing related research. For example, brain-computer diagnosis is not accurate enough and can only distinguish between healthy people and depressed people, but cannot accurately determine the degree of depression. Therefore, there is an urgent need to increase accurate personalized quantitative evaluation indicators to assist clinical diagnosis; treatment methods are risky and have large side effects. Currently, the treatment of depression mainly relies on drug therapy and implantable brain-computer therapy, but different individuals have great differences in their responses to drug treatment, and antidepressants have certain side effects; and implantable BCI usually requires surgery to place electrodes in the skull to collect neural signals, which poses a safety risk of damage to brain tissue. Summary of the invention

[0006] The present invention proposes a method and device for auxiliary identification of depression based on human-machine fusion, which can at least solve one of the technical problems in the background technology.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for assisting depression recognition based on human-computer fusion, specifically comprising the following steps:

[0009] Step 1: Obtain the EEG signal of the target to be tested and preprocess the data;

[0010] Step 2: Perform LZ complexity feature analysis based on the preprocessed data;

[0011] Step 3: Based on the LZ complexity analysis results, a CNN-LSTM confidence analysis model is built in combination with a convolutional neural network (CNN) and a long short-term memory network (LSTM);

[0012] Step 4: Perform confidence analysis and evaluation through the CNN-LSTM confidence analysis model;

[0013] Step 5: The analysis results, LZ complexity and confidence of the EEG signal are transmitted to the human decision-making system to decide whether to adopt the machine classification results and whether to perform TMS treatment;

[0014] Step 6: If TMS treatment is performed, start obtaining the individual head 3D model;

[0015] Step 7: Use the trained stimulation target model to process the individual head 3D model to obtain the individualized stimulation target position;

[0016] Step 8: Filter the EEG signals around the individual stimulation target from the acquired EEG signals, process and predict them, obtain the subject's IAF (Individual Alpha Frequency) and compare them with the normal range to obtain the mapping coefficient, and obtain the individual TMS treatment parameters in combination with the TMS standard treatment parameter range;

[0017] Step 9: Record and save the stimulation parameters, and feed back the stimulation parameters and treatment effects to the decision-making system after the stimulation is completed;

[0018] Step 10: Determine whether to change the treatment plan based on the stimulation parameters and treatment effects.

[0019] Furthermore, the method of acquiring EEG signals and preprocessing the data thereof specifically includes:

[0020] S11. Subject preparation;

[0021] Ensure that the subjects are in a fully relaxed state and instruct them to do deep breathing or relaxation exercises to reduce the myoelectric interference caused by muscle activity;

[0022] Clean the subject's scalp, especially the intended electrode placement area, using a mild skin cleanser to remove oil and impurities. This helps reduce the impedance between the electrode and the scalp and improve signal quality.

[0023] S12, electrode preparation and placement;

[0024] Use conductive gel to coat the electrodes and place them according to the international 10-20 system or the 10-10 system, two commonly used standards for electrode placement, to ensure coverage of different brain areas;

[0025] S13, signal acquisition and conversion;

[0026] Connect the electrodes to the EEG acquisition device, turn on the device, and start recording the electrophysiological activity of the cerebral cortex according to the preset parameters;

[0027] The analog-to-digital converter built into the EEG device converts the captured weak bioelectric signals into digital signals. This process is crucial to the signal fidelity and provides an accurate data basis for subsequent analysis.

[0028] S14, signal preprocessing;

[0029] In EEGLAB, click Re-reference under Tools to select the reference electrode, here select Cz; it should be noted that the low-pass filter is set to 1Hz and the high-pass filter is set to 40Hz; then use the ICA method to decompose the independent EEG and artifact components, and then remove the artifacts, and the rest is the EEG signal without the artifacts; eliminate eye movement components: eye movements include blinking, glancing, and eye movements, etc., which are generally removed using the adjust plug-in in EEGLAB.

[0030] Furthermore, the LZ complexity feature analysis method in step 2 of the present invention is specifically as follows:

[0031] S21, symbolizing the actual EEG signal X;

[0032] S22, data initialization, create a set S containing element s1, and two empty sets Q and SQ, set counter c(n)=1, variable i;

[0033] S23. Perform a traversal operation.

[0034] Furthermore, the traversal operation described in step 2 of the present invention has the following specific operation process:

[0035] S231, traversal starts from the second character of the time sequence, and reads the sequence character to add to the Q set;

[0036] S232, compare the set Q and the set SQ;

[0037] If set Q is a subset of SQ (i.e., SQ contains Q), then directly compare the sizes of i and n. Otherwise, perform an update operation first, add the contents of set Q to set S, clear set Q, and add 1 to c(n).

[0038] S233. Compare the sizes of i and n. If i>n, calculate the LZ complexity equal to c(n) divided by the length of S. Otherwise, continue the traversal operation and loop until the end.

[0039] Furthermore, in step 3 of the present invention, a CNN-LSTM confidence analysis model is constructed, and the specific process is as follows:

[0040] S31. Build a K-fold cross-validation layer to verify the LZ complexity analysis data;

[0041] Perform k-fold cross validation on the entire dataset. In the training set, extract 5% as the validation set, then test the best model on the validation set in the test set, and then test it k times. The generalization error is approximately equal to the average of the k test errors.

[0042] S32. Build a CNN model and design convolutional layers and pooling layers to extract local features of K-fold cross-validation data;

[0043] After the K-fold cross-validation data is input, the convolution, pooling and node expansion operations are performed in sequence;

[0044] Among them, depth-wise separable convolution is used instead of traditional convolution to reduce the number of parameters and improve computational efficiency;

[0045] S33, build LSTM model to capture the dependency of time series data based on the output of CNN model;

[0046] S34, construct an attention layer to normalize the weight parameters of different local features;

[0047] The entire feature is obtained by weighted summing of the weight parameters of different features, and the result is input into a fully connected layer for training. Finally, the output is normalized using the softmax function to obtain the assigned weights of each hidden layer vector and output the result.

[0048] Furthermore, the confidence analysis method of the CNN-LSTM confidence analysis model in step 4 of the present invention is specifically as follows:

[0049] In the confidence analysis using convolutional neural networks, mean square error is used as the cost function; the expected value of the convolutional neural network output is the posterior probability of each indicator category;

[0050] Mean Square Error:

[0051] Where n is the number of samples, y i For real data, To fit the data;

[0052] Commonly used model evaluation indicators for classification problems are classification accuracy Acc, precision P, recall R and F1 value; among them, the F1 value can reflect the precision and recall rate. The larger the F1 value, the better the generalization performance of the classifier.

[0053] The classification accuracy is Acc = (TP + TN) / (TP + TN + FP + FN), the precision is P = TP / (TP + FP), the recall is R = TP / (TP + FN), and the F1 value is F1 = (2 × P × R) / (P + R);

[0054] In the formula: TP means the number of positive classes predicted as positive classes, i.e. true positive classes; FN means the number of positive classes predicted as negative classes, i.e. false negative classes; FP means the number of negative classes predicted as positive classes, i.e. false positive classes; TN means the number of negative classes predicted as negative classes, i.e. true negative classes;

[0055] In the judgment of confidence, the mean square error of accuracy, precision, recall and F1 value is added in the ratio of 5:2:2:1 as the confidence.

[0056] Furthermore, the method for obtaining the stimulation target model in step 7 of the present invention is specifically as follows:

[0057] (1) Obtaining a large number of MRI images and three-dimensional models of the heads of different individual subjects for deep learning network training, wherein the MRI images are collected using a MRI device;

[0058] (2) Import the human brain group atlas, perform image registration on the MRI images, perform individualized target location, and convert them into target coordinates;

[0059] (3) using the individual brain 3D model and individualized targets as training data sets, training the pre-built deep learning network to obtain a trained stimulation target model;

[0060] (4) Through the training of deep learning networks, the three-dimensional model of the individual brain and the relationship between individualized targets are trained;

[0061] (5) Inputting the three-dimensional head model of the target to be tested obtained in step 4 into the trained stimulation target model to perform target prediction, thereby obtaining the target position;

[0062] (6) The TMS coil guided by the neuronavigation system is used to accurately locate the stimulation target of the subject. The neuronavigation system combines MRI and fMRI imaging technology to intuitively guide the placement of the stimulation coil, visualize the magnetic stimulation process, and thus achieve TMS precise stimulation treatment.

[0063] Furthermore, the stimulation process in step 9 of the present invention is specifically as follows:

[0064] Neuronavigation technology is used to guide the TMS stimulation coil to the above-mentioned specific individual stimulation position and IAF is used to determine the stimulation parameters;

[0065] Feedback the target location and stimulation parameters to the decision-making system to confirm whether to perform TMS stimulation;

[0066] After the stimulation, the EEG signal is acquired again, and the treatment effect is analyzed by comparing the degree of depression before and after, and the stimulation position, stimulation parameters and treatment effect are recorded and retained;

[0067] Display the above records and the best TMS treatment positions and parameters, make comprehensive decisions to determine the treatment effect and decide whether to adjust the parameters or continue stimulation.

[0068] Specifically, the beneficial results of the present invention are as follows:

[0069] The LZ complexity of the present invention can be used as an auxiliary algorithm to distinguish normal people from patients with depression. Traditional feature extraction methods all extract time-frequency features, time domain features, frequency domain features and power spectrum features for feature extraction, which is relatively cumbersome and complicated. The present invention focuses on LZ complexity, and LZ complexity can also be used as the input of CNN-LSTM, achieving a seamless connection between the front and the back.

[0070] In the improved k-fold cross-validation method of the present invention, all samples appear once in the test set, that is, there is no problem of insufficient persuasiveness; at the same time, the sample data in the training data set is not significantly reduced, and the obtained model effect will be closer to the real generalization error of the model; in addition, the variable k value can better adapt to the data set, and further improves the accuracy.

[0071] The human-machine fusion diagnosis system of the present invention, which integrates machine intelligence and human decision-making, has achieved the integration and unification of data algorithms and human experience to a certain extent, saved a lot of time used in distinguishing depression, improved diagnostic efficiency, and to a certain extent prevented misdiagnosis caused by over-reliance on algorithm classification results, thereby embodying the scientific and characterization of diagnosis.

[0072] The method for determining individualized TMS targets by integrating three-dimensional deep learning and a neuronavigation system proposed in the present invention uses individual MRI data, human brain group maps, and a three-dimensional head model as training data to train a stimulation target model, which can obtain the precise coordinates of individualized TMS stimulation targets. In practical application, it is only necessary to construct a three-dimensional model of the patient's head to obtain the patient's TMS stimulation targets, thereby reducing the high time and economic costs caused by collecting MRI of the patient's head. By using a variety of data sets in the training stage, the stimulation target model has a wide range of applicability, which enables it to be effectively applied to the TMS individualized stimulation system.

[0073] The TMS individualized stimulation method combined with IAF provided by the present invention obtains the subject's IAF by analyzing and processing the subject's EEG signal, and then analyzes and obtains individualized TMS stimulation parameters. Compared with ordinary TMS treatment, this method can make TMS treatment more in line with the patient's individualized state, thereby enhancing the treatment effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 is a flow chart of the diagnosis and treatment of the present invention;

[0075] Figure 2 This is a specific architecture diagram of the CNN-LSTM confidence analysis model of the present invention;

[0076] Figure 3 A schematic diagram of the steps for calculating LZ complexity of the present invention;

[0077] Figure 4 A flowchart for obtaining a three-dimensional head model of the present invention;

[0078] Figure 5 This is a flowchart of the individualized TMS treatment of the present invention. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0080] Figure 1 It is a flowchart of a human-machine fusion diagnosis system for depression diagnosis method through EEG signal, LZ complexity analysis and CNN-LSTM algorithm, and a treatment system capable of performing personalized TMS, provided by an example of the present invention.

[0081] This example specifically includes the following steps:

[0082] Step 1: Obtain the EEG signal of the target to be tested and preprocess the data;

[0083] Specifically, the acquisition of EEG data of the target to be measured is generally completed using electrodes and EEG signal acquisition equipment. The specific process is as follows:

[0084] S11. Subject preparation;

[0085] Ensure that the subjects are in a fully relaxed state and instruct them to do deep breathing or relaxation exercises to reduce the myoelectric interference caused by muscle activity;

[0086] Clean the subject's scalp, especially the planned electrode placement area, using a mild skin cleanser to remove oil and impurities. This helps reduce the impedance between the electrode and the scalp and improve signal quality.

[0087] S12, electrode preparation and placement;

[0088] Use conductive gel to coat the electrodes and place them according to the international 10-20 system or the 10-10 system, two commonly used standards for electrode placement, to ensure coverage of different brain areas;

[0089] S13, signal acquisition and conversion;

[0090] Connect the electrodes to the EEG acquisition device, turn on the device, and start recording the electrophysiological activity of the cerebral cortex according to the preset parameters;

[0091] The analog-to-digital converter built into the EEG device converts the captured weak bioelectric signals into digital signals. This process is crucial to signal fidelity and provides an accurate data basis for subsequent analysis.

[0092] S14, signal preprocessing;

[0093] In EEGLAB, click Re-reference under Tools to select the reference electrode, here select Cz. It should be noted that the low-pass filter is set to 1Hz and the high-pass filter is set to 40Hz. Then use the ICA method to decompose the independent EEG and artifact components, and then remove the artifacts. What remains is the EEG signal without artifacts. Eliminate eye movement components: Eye movements include blinking, squinting, and eye movements, which are generally removed using the adjust plug-in in EEGLAB.

[0094] Step 2: Perform LZ complexity feature analysis based on the preprocessed data;

[0095] The actual EEG signal is represented by X = {x1, x2, x3, ..., xn}. The complexity cannot directly process the data. The actual EEG signal must be symbolized first, and then the LZ complexity is calculated. The so-called symbolization is the process of converting the actual problem into a mathematical problem through a certain method. The average value method is used to first calculate the average value X0 of X, and compare the elements in X with the elements in X0. If the former is less than or equal to the latter, the corresponding position element in the time series S = {s1, s2, s3, ..., sn} is defined as 0, otherwise it is defined as 1.

[0096] The LZ complexity feature analysis process is as follows Figure 2 As shown, the details are as follows:

[0097] S21, symbolizing the actual EEG signal X;

[0098] S22, data initialization, create a set S containing element s1, and two empty sets Q and SQ, set counter c(n)=1, variable i;

[0099] S23, perform traversal operation;

[0100] The specific operation process of the traversal is as follows:

[0101] S231, traversal starts from the second character of the time sequence, and reads the sequence character to add to the Q set;

[0102] S232, compare the set Q and the set SQ;

[0103] If set Q is a subset of SQ (i.e., SQ contains Q), then directly compare the sizes of i and n. Otherwise, perform an update operation first, add the contents of set Q to set S, clear set Q, and add 1 to c(n).

[0104] S233. Compare the sizes of i and n. If i>n, calculate the LZ complexity equal to c(n) divided by the length of S. Otherwise, continue the traversal operation and loop until the end.

[0105] Specifically, the data preprocessing of this embodiment is completed using the EEGLAB toolbox of MATLAB software. Feature extraction is divided into time domain feature extraction, frequency domain feature extraction and time-frequency nonlinear feature extraction. The present invention focuses on using LZ complexity in time domain features for analysis.

[0106] For the complexity analysis method, the actual EEG signal X is first symbolized to obtain the corresponding time series S. After symbolization, the calculation results of the LZ complexity of each lead are averaged through initialization, traversal, comparison and update to analyze the difference in complexity between the EEG of healthy people and the EEG of depressed patients.

[0107] Step 3: Based on the LZ complexity analysis results, a CNN-LSTM confidence analysis model is built in combination with a convolutional neural network (CNN) and a long short-term memory network (LSTM);

[0108] like Figure 3 The CNN-LSTM confidence analysis model structure shown in the figure has the following construction process:

[0109] S31. Build a K-fold cross-validation layer to verify the LZ complexity analysis data;

[0110] In the CNN-LSTM model training, the present invention uses the improved k-fold cross validation to evaluate the model and verify the results, that is, k-fold cross validation is performed on the entire data set. In the divided training set, 5% is extracted as the validation set, and then the model with the best effect on the validation set is placed in the test set for testing, and then k times are performed, and the generalization error is approximately equal to the average of the k test errors.

[0111] S32. Build a CNN model and design convolutional layers and pooling layers to extract local features of K-fold cross-validation data;

[0112] After the K-fold cross-validation data is input, the convolution, pooling and node expansion operations are performed in sequence;

[0113] Among them, depth-wise separable convolution is used instead of traditional convolution to reduce the number of parameters and improve computational efficiency. The activation function ReLU (Rectified Linear Unit) is used to increase nonlinearity.

[0114] The first layer input dimension (Input_dims) is 7 and the time step (Time_steps) is set to 20.

[0115] The second convolution layer sets kernel_size to 1, uses the Relu function as the activation function, and sets Dropout to 0.5. After pooling and convolution operations, it is input to the LSTM layer.

[0116] S33, build LSTM model to capture the dependency of time series data based on the output of CNN model;

[0117] The output of the CNN model is used as input, and the LSTM layer is designed to capture the long-term dependencies in the time series data. The LSTM layer adopts a 3-layer (64-32-1) LSTM network structure, adds an attention layer after the multi-layer LSTM model, and inputs the result into the fourth layer of attention mechanism.

[0118] In the model training and optimization phase, the data set is divided and the loss function and optimizer are selected for training. At the same time, the model parameters are adjusted and class imbalance is handled.

[0119] S34, construct an attention layer to normalize the weight parameters of different local features;

[0120] The entire feature is obtained by weighted summing of the weight parameters of different features, and the result is input into a fully connected layer for training. Finally, the output is normalized using the softmax function to obtain the assigned weights of each hidden layer vector and output the result.

[0121] The attention layer obtains the entire feature by weighted summation of the weight parameters of different features, inputs the result into a fully connected layer for training, and finally uses the softmax function to normalize the output, obtains the assigned weights of each hidden layer vector, and outputs the result.

[0122] Specifically, step 3 uses the CNN-LSTM confidence analysis model to analyze the EEG signal, uses the improved k-fold cross validation to evaluate the model and verify the results, and feeds back the confidence and original EEG with LZ complexity to the experts for diagnosis and decision-making, including:

[0123] A CNN-LSTM confidence analysis model was built. This model uses multi-layer LSTM, passes the results layer by layer, and inputs them into the attention layer for further training, and performs weighted average output on the output vector of LSTM.

[0124] The improved k-fold cross-validation method is used to divide the data set into training set, test set, and validation set, and put it into the CNN-LSTM confidence analysis model for model training.

[0125] Step 4: Perform confidence analysis and evaluation through the CNN-LSTM confidence analysis model;

[0126] In the confidence analysis using convolutional neural networks, mean square error is used as the cost function. The expected value of the convolutional neural network output is the posterior probability of each indicator category.

[0127] Mean Square Error:

[0128] Where n is the number of samples, y i For real data, To fit the data.

[0129] Commonly used model evaluation indicators for classification problems are classification accuracy (Acc), precision (P), recall (R) and F1 value. Among them, F1 value can reflect precision and recall. The larger the F1 value, the better the generalization performance of the classifier.

[0130] The classification accuracy is Acc = (TP+TN) / (TP+TN+FP+FN), the precision is P = TP / (TP+FP), the recall is R = TP / (TP+FN), and the F1 value is F1 = (2×P×R) / (P+R).

[0131] Where: TP means the number of positive classes predicted as positive classes, i.e. true positive classes; FN means the number of positive classes predicted as negative classes, i.e. false negative classes; FP means the number of negative classes predicted as positive classes, i.e. false positive classes; TN means the number of negative classes predicted as negative classes, i.e. true negative classes.

[0132] In the judgment of confidence, the mean square error of accuracy, precision, recall and F1 value is added in the ratio of 5:2:2:1 as the confidence.

[0133] In the above weight selection, the present invention uses a genetic algorithm to perform optimal weight selection and randomly generate a set of weights, which represent possible solutions to be considered. Each weight represents the contribution of classification accuracy, precision, recall rate and F1 value. Cross-validation is used to evaluate the performance of each weight combination, calculate the fitness of each weight combination, and select a part of the weight combination as the parent generation according to the fitness to generate the next generation weight. The weight with the highest fitness is selected from the final weight combination as the optimal weight combination.

[0134] In step 4, the confidence of the model results is fed back to the experts for diagnosis and decision-making. When the confidence is less than a certain interval, the expert experience is used for analysis and treatment. When the confidence is greater than a certain interval, the expert can analyze and treat according to the classification results, thereby optimizing the diagnosis process. In CNN-LSTM, the model performance is mainly determined based on multiple indicators including precision, sensitivity and accuracy. Because accuracy is the most important value of the classification performance of the convolutional neural network, it represents the correctness of the classification, and the main indicator of the present invention is accuracy.

[0135] In the confidence analysis of CNN-LSTM, the mean square error is used as the cost function, and the expected value of the output is the posterior probability of each model judgment indicator. In the confidence judgment, the ratio and sum of the posterior probabilities of the above model indicators are used as the confidence.

[0136] Step 5: The analysis results, LZ complexity and confidence of the EEG signal are transmitted to the decision-making system to decide whether to adopt the machine's classification results and whether to perform TMS treatment.

[0137] Step 6: If TMS treatment is performed, start obtaining the three-dimensional model of the subject's head;

[0138] The process of obtaining the 3D model is as follows: Figure 4 As shown, the specific process is as follows:

[0139] S61. Take photos;

[0140] Place the subject on a rotating platform and keep the head stable. Take photos of the skull from multiple angles (usually at least dozens of different angles are required). Make sure that the photos from each angle clearly capture the features of the skull. To improve the accuracy of the reconstruction, markers (such as stickers or small balls) can be used to place them at specific locations on the skull during shooting. These markers serve as reference points in the subsequent reconstruction process.

[0141] S62, image processing;

[0142] Import the captured photos into the stereo photography software and perform necessary pre-processing on the photos, such as adjusting brightness, contrast, and cropping, to improve image quality.

[0143] S63, feature point matching;

[0144] Use the algorithms in the stereo photography software to automatically or manually identify and match the same feature points in different photos. Feature point matching is a key step in stereo photography, which determines the accuracy of the 3D model.

[0145] S64, 3D model reconstruction;

[0146] Based on the matched feature points, the software calculates the position of each point in three-dimensional space.

[0147] From these points, the software constructs a preliminary 3D point cloud model. The software then converts this point cloud into a smooth 3D mesh model.

[0148] S65, model optimization;

[0149] The reconstructed 3D model is optimized, including smoothing, noise removal, filling missing parts, feature enhancement, topology optimization, detail adjustment, lighting and texture mapping, and quality inspection. In the smoothing process, in order to eliminate the roughness and discontinuity of the model surface and make the model surface smoother and more natural, the surface vertices of the model are adjusted using the Laplace smoothing algorithm. In the noise removal, the Gaussian filter is used to reduce random errors and unnecessary details in the model and improve the clarity of the model. In the process of filling the missing parts, the Poisson surface reconstruction method is used to repair the areas or holes in the model that are not correctly reconstructed. In addition to the above methods, topology optimization algorithms such as quadrangularization or simplified mesh are also used to reduce the complexity of the model. For this model, if necessary, the model can be manually edited to correct any inaccuracies.

[0150] Final inspection and application: Perform a final inspection on the completed 3D model and obtain the head coordinate set of the 3D model.

[0151] Step 7: Using the trained stimulation target model, process the individual head 3D model to obtain the individualized stimulation target position;

[0152] Among them, the stimulation target model is obtained by training the preset deep learning network using MRI data of different individuals, three-dimensional head models and human brain group maps.

[0153] In step 7, the preset deep learning network is trained using the human brain group map, individual nuclear magnetic resonance data and individual three-dimensional brain image to obtain the stimulation target model. The specific process is as follows:

[0154] Obtain MRI images of the heads of different individuals;

[0155] Use deep learning to register MRI images with human brain tissue maps to obtain individualized TMS stimulation target coordinates;

[0156] Perform 3D reconstruction to obtain accurate images of the head surface and brain tissue of different individuals. This process involves creating a 3D model of the head skin structure and obtaining the precise coordinates of the individual's head.

[0157] The above target coordinates and individual head coordinates are used as training data sets to train the pre-built deep learning network to obtain a trained stimulation target model.

[0158] Specifically, first, it is necessary to use the MRI data of different individuals to train the preset deep learning network to obtain the stimulation target model, which includes:

[0159] (1) Obtain a large number of MRI images and three-dimensional models of the heads of different individual subjects for deep learning network training. The MRI images can be collected by using a MRI, and the method for establishing the three-dimensional head model and obtaining the head coordinate set can be referred to step 4.

[0160] (2) Import the human brain network atlas, perform image registration on the MRI images, and perform individualized target location and conversion into target coordinates. The human brain network atlas was released by the Brain Network Research Center of the Institute of Automation, Chinese Academy of Sciences. Based on the functional connectivity method, the atlas divides the human cerebral cortex and subcortical nuclei into 246 subregions. Through the detailed subregional division of the PFC brain region in the human brain network atlas, the best brain region for stimulation can be selected.

[0161] (3) The individual brain three-dimensional model and individualized targets are used as training data sets to train the pre-built deep learning network to obtain a trained stimulation target model.

[0162] As an implementation method, this embodiment can use the existing U-net network or an improved network based on U-net for training. In practical applications, other network structures can also be used or the network structure can be designed according to actual needs.

[0163] For the specific training process of the network and the related loss function setting, reference may be made to the training process of the existing U-net network, which will not be described in detail in this embodiment.

[0164] (4) Through the above-mentioned training of the deep learning network, a more accurate relationship between the individual brain three-dimensional model and the individualized targets can be trained.

[0165] (5) Then, the three-dimensional model of the target head obtained in step 4 is input into the trained stimulation target model for target prediction, and the target position can be obtained.

[0166] (6) Use the TMS coil guided by the neuronavigation system to accurately locate the stimulation target of the subject. The neuronavigation system can be combined with MRI, fMRI and other imaging technologies to intuitively guide the placement of the stimulation coil, visualize the magnetic stimulation process, and thus achieve TMS precise stimulation treatment.

[0167] Step 8: Filter the EEG signals around the individual stimulation target from the acquired EEG signals, process and predict them, obtain the subject's IAF (Individual Alpha Frequency) and compare them with the normal range to obtain the mapping coefficient, and obtain the individual TMS treatment parameters in combination with the TMS standard treatment parameter range;

[0168] IAF refers to the specific alpha wave frequency range of each person in the EEG. Alpha waves are a type of brain wave produced by the brain in a relaxed, resting state, with a frequency usually between 8 and 13 Hz. IAF is a specific value within this frequency range, which reflects the characteristic frequency of synchronous activity of the cerebral cortex neurons in an individual's resting state. Based on this, magnetic stimulation is triggered, which mainly includes the following processes:

[0169] Independent component analysis is performed on the EEG signal to separate the signals of different channels and obtain independent EEG signals.

[0170] It should be noted that after obtaining the independent EEG signal, it also includes:

[0171] The independent EEG signals were processed to remove electrooculographic artifacts.

[0172] A bandpass filter was applied, retaining signals in the alpha wave frequency range.

[0173] Perform feature extraction to extract the features of the EEG signal through time domain analysis or frequency domain analysis. In frequency domain analysis, Fourier transform (FFT) or wavelet transform is used to analyze the frequency components of the signal.

[0174] Perform feature selection to identify and select the most significant frequency components within the alpha wave frequency range by observing the power spectral density (PSD). Find the alpha wave frequency with the highest power when the individual is in a relaxed state with eyes closed, which is the IAF.

[0175] The IAF calculated above is compared with the normal range to obtain a mapping coefficient, and the standard TMS treatment parameter range is comprehensively calculated with the mapping coefficient to obtain individualized TMS treatment parameters.

[0176] Specifically, the acquired EEG signal is processed and predicted to obtain the IAF, and the time of giving the TMS trigger signal is obtained based on it, including:

[0177] Independent component analysis is performed on the EEG signal to separate signals from different channels, thereby obtaining a target EEG signal corresponding to the individualized target position and EEG signals corresponding to several surrounding positions; the IAF calculated above is compared with the normal range to obtain a mapping coefficient, and the TMS standard treatment parameter range is comprehensively calculated with the mapping coefficient to obtain individualized TMS treatment parameters.

[0178] It should be noted that: in order to increase the excitability of local cortical nerve cells, high-frequency magnetic stimulation is usually used to treat depression. The frequency range is generally selected as 10-20Hz, and this range is defined as the standard treatment frequency range; the stimulation intensity is usually expressed in terms of magnetic induction intensity, and the intensity range is generally selected as 1-3T, and this range is defined as the standard treatment intensity range. The IAF range of normal people is 8-13Hz. The IAF of the subjects is compared with the above range to obtain the mapping coefficient.

[0179] By performing TMS stimulation according to the treatment parameters, the process of performing individualized frequency stimulation at an individualized position can be completed.

[0180] Step 9: Record and save the stimulation parameters, and feed back the stimulation parameters and treatment effects to the decision-making system after the stimulation is completed;

[0181] like Figure 5 The specific process of the stimulation process is as follows:

[0182] During TMS stimulation, stimulation parameters such as stimulation time, stimulation intensity and stimulation frequency were recorded in real time.

[0183] During the TMS stimulation process, the EEG signal of the subject is synchronously acquired. The depressive state of the subject after the stimulation is analyzed through the method in step 5 and compared with the depressive state before the stimulation. The treatment effect is analyzed and fed back to the expert.

[0184] Specifically, the embodiment of the present invention uses neuronavigation technology to guide the TMS stimulation coil to the above-mentioned specific individualized stimulation position and uses IAF to determine the stimulation parameters;

[0185] Feedback the target location and stimulation parameters to humans (such as the attending physician), who then confirm whether to perform TMS stimulation;

[0186] After the stimulation, the EEG signal is acquired again, and the treatment effect is analyzed by comparing the degree of depression before and after, and the stimulation position, stimulation parameters and treatment effect are recorded and retained;

[0187] The above records and the most effective TMS treatment positions and parameters are displayed, and human decision-making is used to determine the treatment effect and decide whether to adjust the parameters or continue stimulation.

[0188] Step 10: Determine whether to change the treatment plan based on the stimulation parameters and treatment effects.

[0189] In summary, the method and system for auxiliary identification of depression based on human-machine fusion of the present invention include acquiring EEG signals, performing complexity analysis on the data after preprocessing; then, analyzing the signals using the CNN-LSTM confidence analysis model, using the improved k-fold cross validation to perform model evaluation and judgment and result verification, and feeding back the confidence and original EEG with LZ complexity obtained by feedback to experts for diagnosis and decision-making; finally, combining deep learning and a neuronavigation system to determine the TMS stimulation target, using IAF to determine the stimulation parameters, performing TMS stimulation and retaining the stimulation position and parameters, and then presenting the stimulation position and parameters to experts for further decision-making and judgment of the treatment effect.

[0190] On the other hand, the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0191] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the depression auxiliary identification methods based on human-computer fusion in the above-mentioned embodiments.

[0192] It is understandable that the system, device and storage medium provided in the embodiments of the present invention correspond to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the above methods.

[0193] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0194] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0195] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0196] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for assisting depression identification based on human-computer fusion, characterized in that: The specific steps include: Step 1: Obtain the EEG signal of the target to be tested and preprocess the data; Step 2: Perform LZ complexity feature analysis based on the preprocessed data; Step 3: Based on the LZ complexity analysis results, a CNN-LSTM confidence analysis model is built by combining the convolutional neural network CNN and the long short-term memory network LSTM; Step 4: Perform confidence analysis and evaluation through the CNN-LSTM confidence analysis model; Step 5: The analysis results, LZ complexity and confidence of the EEG signal are transmitted to the human decision-making system to decide whether to adopt the machine classification results and whether to perform TMS treatment; Step 6: If TMS treatment is performed, start obtaining the individual head 3D model; Step 7: Use the trained stimulation target model to process the individual head 3D model to obtain the individualized stimulation target position; Step 8: Filter the EEG signals around the individual stimulation target from the acquired EEG signals, process and predict them, obtain the subject's IAF (Individual Alpha Frequency), and compare them with the normal range to obtain the mapping coefficient, and combine them with the TMS standard treatment parameter range to obtain the individual TMS treatment parameters; Step 9: Record and save the stimulation parameters, and feed back the stimulation parameters and treatment effects to the decision-making system after the stimulation is completed; Step 10: Determine whether to change the treatment plan based on the stimulation parameters and treatment effects.

2. The method for auxiliary identification of depression based on human-computer fusion according to claim 1 is characterized by: The acquisition of EEG signals and preprocessing of the data specifically include: S11. Subject preparation; Ensure that the subjects are in a fully relaxed state and instruct them to do deep breathing or relaxation exercises to reduce the myoelectric interference caused by muscle activity; Clean the subject's scalp, especially the intended electrode placement area, using a mild skin cleanser to remove oil and impurities. This helps reduce the impedance between the electrode and the scalp and improve signal quality. S12, electrode preparation and placement; Use conductive gel to coat the electrodes and place them according to the international 10-20 system or the 10-10 system, two commonly used standards for electrode placement, to ensure coverage of different brain areas; S13, signal acquisition and conversion; Connect the electrodes to the EEG acquisition device, turn on the device, and start recording the electrophysiological activity of the cerebral cortex according to the preset parameters; The analog-to-digital converter built into the EEG device converts the captured weak bioelectric signals into digital signals. This process is crucial to the signal fidelity and provides an accurate data basis for subsequent analysis. S14, signal preprocessing; In EEGLAB, click Re-reference under Tools to select the reference electrode, here select Cz; it should be noted that the low-pass filter is set to 1Hz and the high-pass filter is set to 40Hz; then use the ICA method to decompose the independent EEG and artifact components, and then remove the artifacts, and the rest is the EEG signal without the artifacts; eliminate eye movement components: eye movements include blinking, glancing, and eye movements, etc., which are generally removed using the adjust plug-in in EEGLAB.

3. The method for auxiliary identification of depression based on human-computer fusion according to claim 1 is characterized in that: The LZ complexity feature analysis method in step 2 is specifically as follows: S21, symbolizing the actual EEG signal X; S22, data initialization, create a set S containing element s1, and two empty sets Q and SQ, set counter c(n)=1, variable i; S23. Perform a traversal operation.

4. The method for auxiliary identification of depression based on human-computer fusion according to claim 3 is characterized in that: The specific operation process of the traversal operation is as follows: S231, traversal starts from the second character of the time sequence, and reads the sequence character to add to the Q set; S232, compare the set Q and the set SQ; If set Q is a subset of SQ, that is, SQ contains Q, then directly compare the sizes of i and n. Otherwise, perform an update operation first, add the contents of set Q to set S, clear set Q, and add 1 to c(n). S233. Compare the sizes of i and n. If i>n, calculate the LZ complexity equal to c(n) divided by the length of S. Otherwise, continue the traversal operation and loop until the end.

5. The method for auxiliary identification of depression based on human-computer fusion according to claim 1 is characterized in that: In step 3, a CNN-LSTM confidence analysis model is built. The specific process is as follows: S31. Build a K-fold cross-validation layer to verify the LZ complexity analysis data; Perform k-fold cross validation on the entire dataset. In the training set, extract 5% as the validation set, then test the best model on the validation set in the test set, and then test it k times. The generalization error is approximately equal to the average of the k test errors. S32. Build a CNN model and design convolutional layers and pooling layers to extract local features of K-fold cross-validation data; After the K-fold cross-validation data is input, the convolution, pooling and node expansion operations are performed in sequence; S33, build LSTM model to capture the dependency of time series data based on the output of CNN model; S34, construct an attention layer to normalize the weight parameters of different local features; The entire feature is obtained by weighted summing of the weight parameters of different features, and the result is input into a fully connected layer for training. Finally, the output is normalized using the softmax function to obtain the assigned weights of each hidden layer vector and output the result.

6. The method for auxiliary identification of depression based on human-computer fusion according to claim 1 is characterized in that: The confidence analysis method of the CNN-LSTM confidence analysis model in step 4 is as follows: In the confidence analysis using convolutional neural networks, mean square error is used as the cost function; the expected value of the convolutional neural network output is the posterior probability of each indicator category; Mean Square Error: Where n is the number of samples, y i For real data, To fit the data; Commonly used model evaluation indicators for classification problems are classification accuracy Acc, precision P, recall R and F1 value; among them, the F1 value can reflect the precision and recall rate. The larger the F1 value, the better the generalization performance of the classifier. The classification accuracy is Acc = (TP + TN) / (TP + TN + FP + FN), the precision is P = TP / (TP + FP), the recall is R = TP / (TP + FN), and the F1 value is F1 = (2 × P × R) / (P + R); In the formula: TP means the number of positive classes predicted as positive classes, i.e. true positive classes; FN means the number of positive classes predicted as negative classes, i.e. false negative classes; FP means the number of negative classes predicted as positive classes, i.e. false positive classes; TN means the number of negative classes predicted as negative classes, i.e. true negative classes; In the judgment of confidence, the mean square error of accuracy, precision, recall and F1 value is added in the ratio of 5:2:2:1 as the confidence.

7. The method for auxiliary identification of depression based on human-computer fusion according to claim 1 is characterized in that: The method for obtaining the stimulation target model in step 7 is as follows: (1) Obtaining a large number of MRI images and three-dimensional models of the heads of different individual subjects for deep learning network training, wherein the MRI images are collected using a MRI device; (2) Import the human brain group atlas, perform image registration on the MRI images, perform individualized target location, and convert them into target coordinates; (3) using the individual brain 3D model and individualized targets as training data sets, training the pre-built deep learning network to obtain a trained stimulation target model; (4) Through the training of deep learning networks, the three-dimensional model of the individual brain and the relationship between individualized targets are trained; (5) Inputting the three-dimensional model of the target head obtained in step 4 into the trained stimulation target model to perform target prediction, thereby obtaining the target position; (6) The TMS coil guided by the neuronavigation system is used to accurately locate the stimulation target of the subject. The neuronavigation system combines MRI and fMRI imaging technology to intuitively guide the placement of the stimulation coil, visualize the magnetic stimulation process, and thus achieve TMS precise stimulation treatment.

8. The method for auxiliary identification of depression based on human-computer fusion according to claim 1 is characterized in that: The stimulation process in step 9 is as follows: During TMS stimulation, stimulation parameters such as stimulation time, stimulation intensity, and stimulation frequency were recorded in real time; During the TMS stimulation process, the EEG signal of the subject is synchronously acquired. The depressive state of the subject after the stimulation is analyzed through the method in step 5 and compared with the depressive state before the stimulation. The treatment effect is analyzed and fed back to the expert.

9. The method for auxiliary identification of depression based on human-computer fusion according to claim 8 is characterized in that: The stimulation process in step 9 is as follows: Neuronavigation technology is used to guide the TMS stimulation coil to the above-mentioned specific individual stimulation position and IAF is used to determine the stimulation parameters; Feedback the target location and stimulation parameters to the decision-making system to confirm whether to perform TMS stimulation; After the stimulation, the EEG signal is acquired again, and the treatment effect is analyzed by comparing the degree of depression before and after, and the stimulation position, stimulation parameters and treatment effect are recorded and retained; Display the above records and the best TMS treatment positions and parameters, make comprehensive decisions to determine the treatment effect and decide whether to adjust the parameters or continue stimulation.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 9.

Images