A cognitive regulation and training system

Abstract

The present invention discloses a cognitive regulation and training system, comprising a cognitive task recommendation module, an information analysis module, a cognitive assessment module, a transcranial magnetic stimulation switch module, a transcranial magnetic control module, and a transcranial magnetic stimulation module, wherein: the cognitive task recommendation module is used to automatically recommend cognitive tasks suitable for patients from a cognitive training knowledge base in the task recommendation module by using an intelligent learning algorithm; the information analysis module is used to analyze brain function information and facial expression information collected by an information collection module; the cognitive assessment module is used to perform comprehensive cognitive assessment; the transcranial magnetic stimulation switch module is used to start or stop magnetic stimulation according to the results of the cognitive task assessment module; and the transcranial magnetic stimulation module is used to start or stop according to a start or stop instruction output by the transcranial magnetic stimulation switch module.

Description

[0001] This application is a divisional application of the invention patent application with application number "202210197647.2", application date March 2, 2022, and invention name "A cognitive regulation and training system based on multimodal fusion". Technical Field

[0002] The present invention relates to the field of cognitive rehabilitation training, and in particular to a cognitive regulation and training system based on multimodal fusion. Background Art

[0003] With the aging of the world's population, the prevalence of cognitive dysfunction in the elderly is gradually increasing. Cognitive dysfunction-related diseases seriously threaten the health of the elderly and bring heavy burdens to families and society. Its prevention and treatment is an important challenge. In recent years, cognitive neuroscience research has provided solid evidence to support the fact that the brain and cognition of elderly people with cognitive impairment are still plastic. Neuroimaging technologies such as magnetic resonance imaging (MRI), electroencephalography (EEG), near-infrared radiofrequency aberration (fNIRS), and functional magnetic resonance imaging (fMRI) have shown broad application prospects in the assessment of cognitive impairment. Exercise intervention can enhance the expression of brain-derived trophic factor (BDNF) and other growth factors that promote neurogenesis, thereby protecting neural structures such as the hippocampus and their corresponding cognitive functions such as memory. New non-invasive neuromodulation technologies such as non-invasive transcranial magnetic stimulation can directly regulate specific brain areas of the cerebral cortex and affect related brain neural pathways, which has a significant positive effect on the cognitive rehabilitation of patients with cognitive impairment. However, each cognitive assessment and training technology has its own advantages and limitations. Therefore, the combination of multimodal cognitive regulation and training technology can truly enhance and prolong cognitive function gains and achieve the optimal cognitive rehabilitation training effect.

[0004] To this end, the present invention provides a cognitive regulation and training system based on multimodal fusion. The transcranial magnetic stimulation neural regulation and cognitive training tasks are multimodally integrated to improve the matching efficiency of brain cognitive function, emotional information feedback and transcranial magnetic stimulation neural regulation, thereby achieving the optimal cognitive rehabilitation feedback training effect. Summary of the invention

[0005] In order to achieve the purpose of the present invention, the following technical solutions are adopted:

[0006] A cognitive regulation and training system based on multimodal fusion, including a cognitive task recommendation module, a cognitive task training module, an information collection module, an information analysis module, a cognitive task adaptive adjustment module, a cognitive assessment module, a transcranial magnetic stimulation switch module, a feedback module, a transcranial magnetic control module, and a transcranial magnetic stimulation module, wherein: the cognitive task recommendation module is used to recommend a list of cognitive tasks suitable for the patient based on the patient's scale score, nuclear magnetic imaging, and clinical diagnosis results; the cognitive task training module is used to display the task on a display; the information collection module is used to collect the patient's brain function information and facial expression information during the cognitive task process; the information analysis module is used to analyze the patient's brain function lateralization, brain function network clustering coefficient, and patient emotions based on the brain function information and facial expression information collected by the information collection module; the cognitive task adaptive adjustment ... recommend a list of cognitive tasks suitable for the patient based on the patient's scale score, nuclear magnetic imaging, and clinical diagnosis results; the cognitive task training module is used to recommend a list of cognitive tasks suitable for the patient based on the patient's scale score, nuclear magnetic imaging, and clinical diagnosis results; the cognitive task training module is used to recommend a list of cognitive tasks suitable for the patient based on the patient's scale score, nuclear magnetic imaging, and clinical diagnosis results; the cognitive task training module is used to recommend a list of cognitive tasks suitable for the patient based on the patient's The adjustment module is used to adaptively adjust the difficulty of cognitive tasks, training time, etc. according to the analysis of the patient's brain function laterality, brain function network clustering coefficient and emotions by the information analysis module; the cognitive assessment module is used to conduct a comprehensive cognitive assessment based on the patient's emotions, brain functions and cognitive task completion; the transcranial magnetic stimulation switch module is used to start or shut down magnetic stimulation according to the results of the cognitive task assessment module; the feedback module is used to feedback the patient's brain function and emotions during the transcranial magnetic stimulation process; the transcranial magnetic stimulation control module is used to adjust the magnetic stimulation control parameters according to the information fed back by the feedback module; the transcranial magnetic stimulation module is used to start or shut down according to the start or shut down instructions output by the transcranial magnetic stimulation switch module, perform non-invasive magnetic stimulation on the patient's brain, and adjust parameters according to the specific parameter instructions output by the transcranial magnetic control module.

[0007] The cognitive regulation and training system based on multimodal fusion, wherein: the cognitive task recommendation module uses an intelligent learning algorithm to automatically recommend cognitive tasks suitable for patients from the cognitive training knowledge base in the task recommendation module.

[0008] The cognitive regulation and training system based on multimodal fusion, wherein the cognitive task recommendation module uses an intelligent learning algorithm to automatically recommend cognitive tasks suitable for patients from the cognitive training knowledge base in the task recommendation module, including:

[0009] Step 1: Preprocess the data in the pre-established cognitive training knowledge base;

[0010] Step 2: Use the convolutional neural network model to extract the data features of the preprocessed MRI brain image;

[0011]

[0012] Among them, conv2 represents the convolution operation, W CNN represents the convolution kernel matrix, valid is the convolution operation type, MRI is the input image matrix, B CNN is the bias, represents the pooling operation;

[0013] Step 3: Input the data features of the magnetic resonance brain image in step 2 and the preprocessed ADAS-Cog scale, biochemical data results and basic information into another convolutional neural network model for training and model optimization to obtain a convolutional neural network-convolutional neural network cognitive task recommendation model:

[0014] [TYPE,DD,TIME,Fe]=F CNN-CNN (Q MRI ,L ADAS-Cog ,CST BT ,J AI )

[0015] Among them, TYPE is the recommended cognitive task type, DD is the difficulty coefficient of the cognitive task, TIME is the training time of the cognitive task, Fe is the training frequency of the cognitive task, and F CNN-CNN is the trained convolutional neural network-convolutional neural network model, L ADAS-Cog is the ADAS-Cog scale data feature after preprocessing, Q MRI is the feature of MRI brain image data, CST BT is the biochemical data feature after preprocessing, J AI It is the basic information data feature after preprocessing.

[0016] The cognitive regulation and training system based on multimodal fusion, wherein: the information analysis module is used to pre-process the cerebral blood oxygen information and facial image information collected by the information collection module, and perform complex wavelet transform and wavelet phase coherence calculation analysis on the pre-processed cerebral blood oxygen information to obtain the patient's laterality index, as follows:

[0017]

[0018] Among them, SI is the laterality index, LK is the total number of nodes in the left brain network, RK is the total number of nodes in the right brain network, b lilj is the number of channels connecting the left brain network node li and the left brain network node lj, q lilj is the shortest path between the left brain network node li and the left brain network node lj, indicating the transmission efficiency, b rirj is the number of channels connecting the right brain network node ri and the right brain network node rj, q rirj is the shortest path between the right brain network node ri and the right brain network node rj, indicating the transmission efficiency, s liki is the number of channels connecting the left brain network node li with other left and right brain network nodes ki, s rikjis the number of channels connecting the right brain network node ri with other left and right brain network nodes kj, λ is the correction coefficient; nodes li and lj are different nodes in the left brain network; node ri and node rj are different nodes in the right brain network.

[0019] The cognitive regulation and training system based on multimodal fusion, wherein:

[0020] When the patient is left-handed, When the patient is right-handed, in:

[0021]

[0022]

[0023] The shortest path between the left brain network node li and the left brain network node lj is:

[0024]

[0025] Wherein, N is the number of brain blood oxygen signals contained in the left brain blood oxygen signal data sequence after complex wavelet transform, is the phase difference of the cerebral blood oxygen signal between node li and node lj;

[0026] The shortest path between the right brain network node ri and the right brain network node rj is:

[0027]

[0028] Wherein, M is the number of cerebral blood oxygen signal data sequences after complex wavelet transform. is the phase difference of cerebral blood oxygen signal between node ri and node rj;

[0029] The cognitive regulation and training system based on multimodal fusion, wherein: the information analysis module is used for brain function connection analysis based on graph theory to obtain the patient's brain function network clustering coefficient:

[0030]

[0031] Where CI is the clustering coefficient of the brain function network, N is the total number of nodes in the brain network (N = LK + RK), k i is the total number of connections between brain network node i and other nodes, λ ij is the number of channels connecting node i to node j, λ ij (h) is the number of nodes h that are not nodes i and j in the shortest path between nodes i and j, and d ij is the shortest path between node i and node j.

[0032] The cognitive regulation and training system based on multimodal fusion, wherein: the information analysis module is used to extract features from the pre-processed facial image information using a convolutional neural network algorithm to obtain the patient's facial expression coefficient:

[0033] Y q =G CNN (W,D feature ,B)

[0034] Among them, G CNN is the convolutional neural network model, W is the connection weight of different convolutional layers, D feature is the extracted facial image features, B is the bias of the convolutional neural network model, and Y q The expression coefficient of the patient is obtained, including three emotion types: positive, neutral and negative.

[0035] The cognitive regulation and training system based on multimodal fusion, wherein: the cognitive assessment module is used to perform a comprehensive cognitive assessment based on the patient's emotions, brain function laterality, and cognitive task completion.

[0036] Q=C1*Y q +C2*SI+C3*CI+C4*P n

[0037] Among them, Q is the comprehensive cognitive evaluation value, Y q is the emotion index, SI is the brain function laterality index, CI is the brain function network clustering coefficient, P n It is the cognitive task score automatically calculated by the cognitive task training module according to the task difficulty coefficient and task time during the cognitive training execution period. C1, C2, C3, and C4 are weight coefficients. q The value is -1; when the patient is in a positive mood, Y q The value is 1; when the patient is emotionally neutral, Y q The value is 0.

[0038] The cognitive regulation and training system based on multimodal fusion, wherein: when a patient undergoes transcranial magnetic stimulation for the first time, the transcranial magnetic stimulation control module finds the information closest to the current patient information from the transcranial magnetic stimulation parameter database, including: converting the text information of the current patient and the patient in the transcranial magnetic stimulation parameter database, brain function laterality index, brain function network clustering coefficient index, emotion index, maximum stimulation frequency, and maximum stimulation intensity into vector data through word2vec, and calculating the vector similarity, and taking the vector similarity as the patient with the closest information to the current patient;

[0039] The calculation formula for vector similarity is:

[0040]

[0041] Where c(x,y) represents the vector similarity between the current patient x and the patient y in the transcranial magnetic stimulation parameter database, n is the number of vector data after patient information conversion, D x (i) represents the i-th data vector of the current patient x, D y (i) represents the i-th data vector of patient y in the transcranial magnetic stimulation parameter database, ||D x (i) || represents the modulus of the i-th data vector of the current patient x, · represents vector multiplication, i.e., the scalar product of the vectors, ||D y (i)|| represents the modulus of the i-th data vector of patient y in the transcranial magnetic stimulation parameter database, k i Represents the weight of the i-th data vector, satisfying g xy Indicates the number of vectors that are the same between the current patient x and the patient y in the transcranial magnetic stimulation parameter database. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the structure of the cognitive regulation and training system based on multimodal fusion of the present invention;

[0043] Figure 2 This is a flow chart of the multimodal fusion cognitive regulation and training method of the present invention;

[0044] Figure 3 This is a schematic diagram of transcranial magnetic stimulation parameter adjustment according to the present invention. DETAILED DESCRIPTION

[0045] The following is combined with Figure 1 , the specific implementation methods of the present invention are described in detail.

[0046] like Figure 1As shown, a cognitive regulation and training system based on multimodal fusion of the present invention includes a cognitive task recommendation module, a cognitive task training module, an information collection module, an information analysis module, a cognitive task adaptive adjustment module, a cognitive assessment module, a transcranial magnetic stimulation switch module, a feedback module, a transcranial magnetic control module, and a transcranial magnetic stimulation module, wherein: the cognitive task recommendation module is used to recommend a list of cognitive tasks suitable for the patient based on the patient's ADAS-Cog scale score, nuclear magnetic imaging and clinical diagnosis results using an artificial intelligence algorithm; the information collection module is used to collect the patient's brain function information and facial expression information during the cognitive task process; the information analysis module is used to analyze the patient's brain function lateralization, brain function network clustering coefficient and patient emotion based on the brain function information and facial expression information collected by the information collection module; the cognitive task adaptive adjustment ... cognitive task adaptive adjustment module is used to recommend a list of cognitive tasks suitable for the patient based on the patient's ADAS-Cog scale The adaptive adjustment module is used to adaptively adjust the difficulty of cognitive tasks, training time, etc. according to the analysis of the patient's brain function laterality, brain function network clustering coefficient and emotions by the information analysis module; the cognitive assessment module is used to conduct comprehensive cognitive assessment according to the patient's emotions, brain functions and cognitive task completion; the transcranial magnetic stimulation switch module is used to start or shut down magnetic stimulation according to the results of the cognitive task assessment module; the feedback module is used to feedback the patient's brain function and emotions during the transcranial magnetic stimulation process; the transcranial magnetic stimulation control module is used to adjust the magnetic stimulation control parameters according to the information fed back by the feedback module; the transcranial magnetic stimulation module is used to start or shut down according to the start or shut down instructions output by the transcranial magnetic stimulation switch module, to perform non-invasive magnetic stimulation on the patient's brain, and can adjust parameters according to the specific parameter instructions output by the transcranial magnetic control module.

[0047] The cognitive task recommendation module uses intelligent learning algorithms to automatically recommend cognitive tasks suitable for patients from the cognitive training knowledge base in the task recommendation module. The cognitive training knowledge base is composed of clinical patients' ADAS-Cog scale (language comprehension ability, recall test instructions, attention, intentionality, etc.) scores, MRI images and biochemical results, and cognitive training programs prescribed by doctors, and is continuously updated according to the adjustment of clinical cognitive training tasks.

[0048] Preferably, the types of cognitive tasks include eye movement tasks, balance tasks, memory tasks, attention tasks, etc. For example, eye movement tasks include following training, directional saccade training, reverse saccade training, etc.; balance tasks include sitting balance training, left-right balance training, head balance training, etc.; memory tasks include digital memory training, picture memory training, color memory training, sentence repetition training, etc.; attention tasks include calculation training, expression training, game command training, etc. under different sound and light stimulations.

[0049] Furthermore, each cognitive training task is designed to gradually increase in difficulty from easy to difficult.

[0050] The cognitive task training module is used to display cognitive tasks on a display for users to perform cognitive task training.

[0051] The intelligent learning algorithm is based on the cognitive training knowledge base and establishes a convolutional neural network-convolutional neural network model to recommend the type, difficulty coefficient and task time of cognitive training tasks for patients. The specific method is as follows:

[0052] Step 1: Preprocess the data in the pre-established cognitive training knowledge base: filter and denoise the MRI brain images, and normalize and preprocess the ADAS-Cog scale, biochemical data results (such as Aβ42 and Tau cerebrospinal fluid proteins, serum adiponectin levels, high-sensitivity C-reactive protein, etc.) and basic information (age, gender, education, occupation, history of hypertension, history of diabetes, and other personal information);

[0053] Step 2: Use the convolutional neural network model to extract the data features of the preprocessed MRI brain image;

[0054]

[0055] Among them, conv2 represents the convolution operation, W CNN represents the convolution kernel matrix, valid is the convolution operation type, MRI is the input image matrix, B CNN is the bias, represents the pooling operation;

[0056] Step 3: Input the data features of the magnetic resonance brain image in step 2 and the pre-processed ADAS-Cog scale, biochemical data results and basic information into another convolutional neural network model for training and model optimization, and continuously optimize the model parameters to finally obtain a convolutional neural network-convolutional neural network cognitive task recommendation model.

[0057] [TYPE,DD,TIME,Fe]=F CNN-CNN (Q MRI ,L ADAS-Cog ,CST BT ,J AI )

[0058] Among them, TYPE is the recommended cognitive task type, DD is the difficulty coefficient of the cognitive task, TIME is the training time of the cognitive task, Fe is the training frequency of the cognitive task, and F CNN-CNN is the trained convolutional neural network-convolutional neural network model, L ADAS-Cog is the ADAS-Cog scale data feature after preprocessing, Q MRI is the feature of MRI brain image data, CST BT is the biochemical data feature after preprocessing, JAI It is the basic information data feature after preprocessing.

[0059] Preferably, the patient's ADAS-Cog scale (language comprehension ability, recall test instructions, attention, intentionality, etc.) scores, MRI images and biochemical results will be evaluated every 2 months, and the type, difficulty coefficient and task time of cognitive tasks will be re-recommended to the patient through the intelligent learning algorithm.

[0060] The information acquisition module is used to collect the brain function information and facial image information of the patient during the cognitive task. The near-infrared brain function device is used to collect the brain blood oxygen information of the patient during the cognitive training task; the depth camera is used to collect the facial image information of the patient during the cognitive training task.

[0061] The information analysis module is used to pre-process the cerebral blood oxygen information and facial image information collected by the information acquisition module, and perform complex wavelet transform and wavelet phase coherence calculation and analysis on the pre-processed cerebral blood oxygen information to obtain the patient's laterality index, as follows:

[0062]

[0063] Among them, SI is the laterality index, LK is the total number of nodes in the left brain network, RK is the total number of nodes in the right brain network, b lilj is the number of channels connecting the left brain network node li and the left brain network node lj, q lilj is the shortest path between the left brain network node li and the left brain network node lj, indicating the transmission efficiency, b rirj is the number of channels connecting the right brain network node ri and the right brain network node rj, q rirj is the shortest path between the right brain network node ri and the right brain network node rj, indicating the transmission efficiency, s liki is the number of channels connecting the left brain network node li with other left and right brain network nodes ki, s rikj is the number of channels connecting the right brain network node ri with other left and right brain network nodes kj, λ is the correction coefficient; nodes li and lj are different nodes in the left brain network; node ri and node rj are different nodes in the right brain network.

[0064] Furthermore, when the patient is left-handed, When the patient is right-handed, in:

[0065]

[0066]

[0067] The shortest path between the left brain network node li and the left brain network node lj is:

[0068]

[0069] Wherein, N is the number of brain blood oxygen signals contained in the left brain blood oxygen signal data sequence after complex wavelet transform, is the phase difference of the cerebral blood oxygen signal between node li and node lj.

[0070] The shortest path between the right brain network node ri and the right brain network node rj is:

[0071]

[0072] Wherein, M is the number of cerebral blood oxygen signal data sequences after complex wavelet transform. is the phase difference of the cerebral blood oxygen signal between node ri and node rj.

[0073] And the brain functional connection analysis based on graph theory, the patient's brain functional network clustering coefficient was obtained:

[0074]

[0075] Where CI is the clustering coefficient of the brain function network, N is the total number of nodes in the brain network (N = LK + RK), k i is the total number of connections between brain network node i and other nodes, λ ij is the number of channels connecting node i to node j, λ ij (h) is the number of nodes h that are not nodes i and j in the shortest path between nodes i and j, and d ij is the shortest path between node i and node j.

[0076] The clustering coefficient of the brain function network is used to measure the degree to which nodes in the brain network tend to cluster together with nearby nodes. Studies have shown that as patients' cognitive abilities decline, the clustering coefficient of the prefrontal brain function network also decreases.

[0077] The preprocessed facial image information is subjected to feature extraction using a convolutional neural network algorithm to obtain the patient’s facial expression coefficient:

[0078] Y q =G CNN (W,D feature ,B)

[0079] Among them, G CNN is the convolutional neural network model, W is the connection weight of different convolutional layers, D feature is the extracted facial image features, B is the bias of the convolutional neural network model, and Yq The patient's expression coefficient is divided into three types: positive, neutral, and negative. q The value [0.6, 0.9) indicates positive emotion, the value [0.3, 0.6) indicates neutral emotion, and the value (0, 0.3) indicates negative emotion.

[0080] The cognitive task adaptive adjustment module is used to adaptively adjust the cognitive task difficulty, training time, etc. according to the patient's brain function lateralization, brain function network clustering coefficient and emotional type analyzed by the information analysis module. The specific adjustments are as follows:

[0081] When the patient's mood is negative, the type of cognitive task is changed; when the patient's mood is positive and the brain function laterality is lower than the threshold, the difficulty of the cognitive task is lowered; when the patient's mood is positive and the brain function laterality is higher than the threshold, the difficulty of the cognitive task is increased; when the patient's mood is positive and the brain function network clustering coefficient is lower than the threshold, the cognitive task time is lowered; when the patient's mood is positive and the brain function network clustering coefficient is lower than the threshold, the cognitive task time is increased.

[0082] Optionally, the patient's emotional state, brain function laterality, brain function network clustering coefficient, and corresponding cognitive training task files are saved. If the patient is not undergoing cognitive training for the first time, training begins with the middle value of the difficulty coefficient of the cognitive training task of the previous day; if the patient is undergoing cognitive task training for the first time, training begins with the cognitive task recommended by the cognitive task recommendation module.

[0083] The cognitive assessment module is used to conduct a comprehensive cognitive assessment based on the patient's mood, brain function laterality, and cognitive task completion.

[0084] Q=C1*Y q +C2*SI+C3*CI+C4*P n

[0085] Among them, Q is the comprehensive cognitive evaluation value, Y q is the emotion index, SI is the brain function laterality index, CI is the brain function network clustering coefficient, P n It is the cognitive task score automatically calculated by the cognitive task training module according to the task difficulty coefficient and task time during the cognitive training execution period. C1, C2, C3, and C4 are weight coefficients. q The value is -1; when the patient is in a positive mood, Y q The value is 1; when the patient is emotionally neutral, Y q The value is 0.

[0086] The transcranial magnetic stimulation switch module is used to start or turn off magnetic stimulation according to the results of the cognitive task evaluation module. When the comprehensive cognitive evaluation value Q obtained by the cognitive evaluation module is lower than the threshold value 0.6, the transcranial magnetic stimulation switch module is turned on to perform transcranial magnetic stimulation on the patient.

[0087] The feedback module is used to receive the emotion index, brain function lateralization and brain function network clustering coefficient obtained by the information analysis module during the transcranial magnetic stimulation process, and feed back the received information to the transcranial magnetic stimulation control module;

[0088] The transcranial magnetic stimulation control module is used to adjust the magnetic stimulation control parameters according to the information fed back by the feedback module;

[0089] Studies have shown that when the intensity and location of magnetic stimulation remain unchanged, magnetic stimulation of different frequencies has a certain correlation with the laterality of brain function in patients with cognitive dysfunction; when the frequency and intensity of magnetic stimulation remain unchanged, magnetic stimulation of different locations has a certain correlation with the clustering coefficient of the patient's brain functional network; when the frequency and location of magnetic stimulation remain unchanged, magnetic stimulation of different intensities has a certain effect on the emotions of patients with cognitive dysfunction.

[0090] like Figure 3 As shown, the specific adjustments are as follows: when the patient uses the transcranial magnetic stimulation module for the first time, the transcranial magnetic stimulation control module pre-sets the frequency, intensity, and stimulation site of the transcranial magnetic stimulation module, and determines whether the brain function laterality index SI exceeds the threshold value M1. If not, the frequency of the transcranial magnetic stimulation module is increased; if it exceeds the normal threshold, the frequency of the transcranial magnetic stimulation module is lowered; then determine whether the brain function network clustering coefficient index CI exceeds the threshold value M2. If not, the stimulation site of the transcranial magnetic stimulation module remains unchanged; if it exceeds the normal threshold, the stimulation site of the transcranial magnetic stimulation module is adjusted; then determine whether the patient's emotional index Y q Whether it is a negative emotion, if so, the intensity of the transcranial magnetic stimulation module is increased; if not, the intensity of the transcranial magnetic stimulation module remains unchanged.

[0091] The transcranial magnetic stimulation module is used to start or shut down according to the start or shut down instruction output by the transcranial magnetic stimulation switch module, and to adjust parameters according to the specific parameter instructions output by the transcranial magnetic control module.

[0092] The first preset transcranial magnetic stimulation time in the cognitive training task is 10 minutes, followed by a 2-minute break and then cognitive task training. When the cognitive task training is performed again, if the patient's comprehensive cognitive assessment value Q can reach more than 60%, it means that the transcranial magnetic stimulation effect is obvious. If the patient's comprehensive cognitive assessment value Q is still below 60%, the time of the next transcranial magnetic stimulation will be increased, for example, by 2 minutes each time.

[0093] After the transcranial magnetic stimulation module is executed during the cognitive training process, the transcranial magnetic stimulation frequency, intensity, stimulation site and other parameters with the highest cognitive evaluation coefficient of the patient are recorded and saved as the initial values ​​of the transcranial magnetic stimulation parameters for the next cognitive training.

[0094] The threshold value M1 of the brain function laterality index is mainly determined by the maximum frequency that the patient can withstand when receiving transcranial magnetic stimulation in the resting state before each cognitive training; the threshold value M2 of the brain function network clustering coefficient index is mainly determined by the maximum value of the brain function network clustering coefficient calculated when the patient's left and right brain areas are subjected to resting-state transcranial magnetic stimulation before each cognitive training.

[0095] The maximum frequency and intensity of transcranial magnetic stimulation should not exceed the maximum frequency and intensity of transcranial magnetic stimulation that the patient can withstand when in a resting state.

[0096] Furthermore, the transcranial magnetic stimulation control module continuously collects data indicators such as basic patient information, brain function laterality index, brain function network clustering coefficient index, emotion index, maximum stimulation frequency, maximum stimulation intensity and corresponding optimal transcranial magnetic stimulation parameters to establish a transcranial magnetic stimulation parameter database.

[0097] When a patient undergoes transcranial magnetic stimulation for the first time, the information closest to the current patient's information is found from the transcranial magnetic stimulation parameter database. The specific method is: the text information of the current patient and the patient in the transcranial magnetic stimulation parameter database, brain function laterality index, brain function network clustering coefficient index, emotion index, maximum stimulation frequency, and maximum stimulation intensity are converted into vector data through word2vec, and the vector similarity is calculated, and the vector similarity is used as the patient with the closest information to the current patient. The calculation formula for vector similarity is:

[0098]

[0099] Where c(x,y) represents the vector similarity between the current patient x and the patient y in the transcranial magnetic stimulation parameter database, n is the number of vector data after patient information conversion, D x (i) represents the i-th data vector of the current patient x, D y (i) represents the i-th data vector of patient y in the transcranial magnetic stimulation parameter database, ||D x (i) || represents the modulus of the i-th data vector of the current patient x, · represents vector multiplication, i.e., the scalar product of the vectors, ||D y (i)|| represents the modulus of the i-th data vector of patient y in the transcranial magnetic stimulation parameter database, k i Represents the weight of the i-th data vector, satisfying g xyIndicates the number of vectors that are the same between the current patient x and the patient y in the transcranial magnetic stimulation parameter database.

[0100] When the value of c(x,y) is 0.9-1, it is considered that the similarity between patient y and the current patient x is the highest; when the value of c(x,y) is 0-0.2, it is considered that there is no similarity between patient y and the current patient x.

[0101] If multiple patients most similar to the current patient are obtained from the transcranial magnetic stimulation parameter database, the patient with the lowest transcranial magnetic stimulation intensity will be used as the initial transcranial magnetic stimulation plan for the current patient.

[0102] If the patient most similar to the current patient is not found in the transcranial magnetic stimulation parameter database, the transcranial magnetic stimulation parameters can be intelligently derived by establishing a deep learning model. The specific method is:

[0103] Step 1: The transcranial magnetic stimulation parameter database was randomly divided into a training set and a test set according to a 4:2 ratio;

[0104] Step 2: Initialize the long short-term memory neural network LSTM model and continuously train the model with the training set. After every 1,000 training times, use the test set to test and calculate the accuracy of the LSTM model.

[0105] Step 3: When the accuracy of the LSTM model reaches more than 85%, the model validation optimization is performed to obtain the transcranial magnetic stimulation parameter R of the current patient through the LSTM model. n (x), and the transcranial magnetic stimulation parameter Q given by the doctor n (x) The square root of the difference is used as the loss function of the LSTM model, that is, the loss function is

[0106] Step 4: The LSTM model parameters corresponding to the minimum loss function are the final LSTM deep learning model, and the patient's transcranial magnetic stimulation parameters are calculated through this model.

[0107] The cognitive training system also includes a display device, such as a display screen, for presenting the feedback process of the feedback module in real time.

[0108] Compared with the prior art, the beneficial effects of the present invention are:

[0109] (1) A comprehensive cognitive assessment model for patients is established through the laterality of brain function, the clustering coefficient of brain function network, emotions, and cognitive task completion, taking into full account the fusion of multiple sources of information such as brain, emotions, and cognitive tasks during the patient's cognitive training process, so as to enable real-time and accurate assessment of the patient's cognitive function.

[0110] (2) Make full use of the changes in the information interaction characteristics of characteristic indicators such as the patient's brain function laterality, brain function network clustering coefficient, and patient emotions to adjust the patient's transcranial magnetic stimulation parameters in real time, thereby promoting the coordinated optimization and real-time feedback of the brain and transcranial magnetic stimulation data.

[0111] (3) This system can be used to provide patients with a cognitive function training program that integrates personalized adaptive cognitive tasks and transcranial magnetic neural regulation multimodality, thereby improving the efficiency and effectiveness of patients' cognitive function rehabilitation.

[0112] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, it should be understood by those skilled in the art that any person familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or perform equivalent replacements on some of the technical features, and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A cognitive regulation and training system, comprising a cognitive task recommendation module, an information analysis module, a cognitive assessment module, a transcranial magnetic stimulation switch module, a transcranial magnetic stimulation control module, a transcranial magnetic stimulation module, a cognitive task training module, an information collection module, a feedback module, and a cognitive task adaptive adjustment module, characterized in that: The cognitive task recommendation module is used to automatically recommend cognitive tasks suitable for patients from the cognitive training knowledge base in the task recommendation module using an intelligent learning algorithm; the information analysis module is used to analyze the patient's brain function laterality, brain function network clustering coefficient and patient emotions based on the brain function information and facial expression information collected by the information acquisition module; the cognitive assessment module is used to perform comprehensive cognitive assessment; the transcranial magnetic stimulation switch module is used to start or shut down magnetic stimulation according to the results of the cognitive task assessment module; the transcranial magnetic stimulation module is used to start or shut down according to the start or shut down instructions output by the transcranial magnetic stimulation switch module; the information acquisition module is used to collect the patient's brain function information and facial expression information during the cognitive task process; the feedback module is used to feedback the patient's brain function and emotional status during the transcranial magnetic stimulation process; the cognitive task training module is used to display cognitive tasks on a display for users to perform cognitive task training; the cognitive task adaptive adjustment module is used to perform adaptive adjustment of cognitive task difficulty and training time based on the patient's brain function laterality, brain function network clustering coefficient and emotions analyzed by the information analysis module; the transcranial magnetic stimulation control module is used to adjust the magnetic stimulation control parameters based on the information fed back by the feedback module; The cognitive task recommendation module uses intelligent learning algorithms to automatically recommend cognitive tasks suitable for patients from the cognitive training knowledge base in the task recommendation module, including: Step 1: Preprocess the data in the pre-established cognitive training knowledge base; Step 2: Use the convolutional neural network model to extract the data features of the preprocessed MRI brain image; Among them, conv2 represents the convolution operation, W CNN represents the convolution kernel matrix, valid is the convolution operation type, MRI is the input image matrix, B CNN is the bias, represents the pooling operation; Step 3: Input the data features of the magnetic resonance brain image in step 2 and the preprocessed ADAS-Cog scale, biochemical data results and basic information into another convolutional neural network model for training and model optimization to obtain a convolutional neural network-convolutional neural network cognitive task recommendation model: Among them, TYPE is the recommended cognitive task type, DD is the difficulty coefficient of the cognitive task, TIME is the training time of the cognitive task, Fe is the training frequency of the cognitive task, and F CNN-CNN is the trained convolutional neural network-convolutional neural network model, L ADAS-Cog is the ADAS-Cog scale data feature after preprocessing, Q MRI is the feature of MRI brain image data, CST BT is the biochemical data feature after preprocessing, and the biochemical data feature includes Aβ42 and Tau cerebrospinal fluid protein, serum adiponectin level, and high-sensitivity C-reactive protein; J AI The basic information data features after preprocessing include age, gender, education background, occupation, history of hypertension, and history of diabetes.

2. The cognitive regulation and training system according to claim 1, characterized in that: The information analysis module is used to pre-process the brain blood oxygen information and facial image information collected by the information collection module.

3. The cognitive regulation and training system according to claim 1, characterized in that: The information analysis module is used to pre-process the cerebral blood oxygen information and facial image information collected by the information acquisition module, and perform complex wavelet transform and wavelet phase coherence calculation analysis on the pre-processed cerebral blood oxygen information to obtain the patient's laterality index, as follows: Among them, SI is the laterality index, LK is the total number of nodes in the left brain network, and RK is the total number of nodes in the right brain network. is the number of channels connecting the left brain network node li and the left brain network node lj, is the shortest path between the left brain network node li and the left brain network node lj, indicating the transmission efficiency, is the number of channels connecting the right brain network node ri and the right brain network node rj, is the shortest path between the right brain network node ri and the right brain network node rj, indicating the transmission efficiency, is the number of channels connecting the left brain network node li with other left and right brain network nodes ki, is the number of channels connecting the right brain network node ri with other left and right brain network nodes kj, is the correction coefficient; nodes li and lj are different nodes in the left brain network; nodes ri and rj are different nodes in the right brain network.

4. The cognitive regulation and training system according to claim 3, characterized in that: When the patient is left-handed, , when the patient is right-handed, ,in: The shortest path between the left brain network node li and the left brain network node lj is: Wherein, N is the number of brain blood oxygen signals contained in the left brain blood oxygen signal data sequence after complex wavelet transform, is the phase difference of the cerebral blood oxygen signal between node li and node lj; The shortest path between the right brain network node ri and the right brain network node rj is: Wherein, M is the number of cerebral blood oxygen signal data sequences after complex wavelet transform. is the phase difference of the cerebral blood oxygen signal between node ri and node rj.

5. The cognitive regulation and training system according to claim 3, characterized in that: The information analysis module is used for brain function connection analysis based on graph theory to obtain the patient's brain function network clustering coefficient: Among them, CI is the clustering coefficient of brain function network, N is the total number of nodes in the brain network, N=LK+RK, k i is the total number of connections between brain network node i and other nodes, is the number of channels connecting node i to node j, is the number of nodes h that are not nodes i and j in the shortest path between nodes i and j. is the shortest path between node i and node j.

Images