A transcranial light stimulation regulation method and system

Abstract

The application provides a transcranial light stimulation regulation method and system, wherein a brain light field model module is used to construct a target multi-classifier, and the target multi-classifier is used to output transcranial light stimulation parameter information based on a target patient data set; a virtual reality module is used to provide a virtual reality rehabilitation training task for a user; a light emission control module is used to adjust the parameters of a light stimulation module according to the output transcranial light stimulation parameters, and the light stimulation module is used to stimulate the patient with light; an information acquisition and analysis module is used to acquire and analyze the eye movement and facial video information of the user during the virtual reality rehabilitation training task during the transcranial light stimulation process; a brain function evaluation module is used to evaluate the brain function of the patient before and after the transcranial light regulation; and a feedback module is used to feed back the eye movement and facial video information output by the information acquisition and analysis module and the brain function information output by the brain function evaluation module to the light emission control module for adjustment of the light stimulation parameters, so as to improve the transcranial light stimulation regulation efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical equipment, and in particular to a transcranial light stimulation regulation method and system. BACKGROUND

[0002] With the aging of the world population, the prevalence of brain function disorders such as stroke and cognitive impairment is gradually increasing, which seriously threatens the health of the elderly and brings heavy burden to families and society. Its prevention and treatment is an important challenge. Transcranial light, as a new type of non-invasive brain nerve stimulation regulation technology, penetrates the skull by low-intensity infrared light (wavelength 625-740nm) or near-infrared light (wavelength 750-1100nm) to regulate brain neurons, and has the characteristics of high safety, high adaptability and non-invasiveness, which has attracted more and more attention in the field of neural regulation.

[0003] How to determine the transcranial light parameters used in transcranial light stimulation regulation to achieve better regulation effect is a very important problem. SUMMARY

[0004] Therefore, the embodiments of the present application provide a transcranial light stimulation regulation method and system, which can timely and accurately adjust the parameters of the light emitting unit and improve the transcranial light stimulation regulation efficiency.

[0005] According to an aspect of the present application, a transcranial light stimulation regulation method is provided, the method comprising:

[0006] Obtaining historical sample data of a plurality of brain function disorder patients in a historical record and sample labels of the historical sample data to obtain a target training set;

[0007] Obtaining a plurality of target patient data of a target patient to be subjected to transcranial light stimulation to obtain a target patient data set;

[0008] Training a target multi-classifier based on each piece of historical sample data and sample labels in the target training set, the target multi-classifier being used to output transcranial light stimulation parameters;

[0009] Inputting the target patient data set into the target multi-classifier to enable the target multi-classifier to output target transcranial light stimulation parameters of the target patient based on the target patient data set;

[0010] Using the target transcranial light stimulation parameters to regulate the target patient by transcranial light stimulation, and collecting target brain function feedback information, target eye movement information and target facial video information of the target patient during the transcranial light stimulation regulation process;

[0011] obtain a target visual-spatial attention index of the target patient based on the target eye movement information and a preset corresponding relationship between eye movement information and the visual-spatial attention index; and obtain a target emotion index of the target patient based on the target facial video information by using a pre-trained convolutional neural network;

[0012] adjust the target transcranial light stimulation parameter based on the target brain function feedback information, the target visual-spatial attention index, and the target emotion index of the target patient;

[0013] adjust a light emission parameter of the light emission unit according to the target transcranial light stimulation regulation parameter, the light emission parameter including one or more of a wavelength, a frequency, a pulse width, and an emission time of the light.

[0014] According to another aspect of the present application, a transcranial light stimulation regulation system is provided, the system comprising:

[0015] a first acquisition module configured to acquire historical sample data of a plurality of brain dysfunction patients in historical records and sample labels of the historical sample data, and obtain a target training set;

[0016] a second acquisition module configured to acquire a plurality of target patient data of a target patient to be subjected to transcranial light stimulation regulation, and obtain a target patient data set;

[0017] a classifier training module configured to train a target multi-classifier based on each piece of the historical sample data and the sample labels in the target training set, the target multi-classifier being configured to output a transcranial light stimulation parameter;

[0018] a parameter output module configured to input the target patient data set to the target multi-classifier, so that the target multi-classifier outputs a target transcranial light stimulation parameter of the target patient based on the target patient data set;

[0019] an information acquisition module configured to perform transcranial light stimulation regulation on the target patient by using the target transcranial light stimulation parameter, and acquire target brain function feedback information, target eye movement information, and target facial video information of the target patient during the transcranial light stimulation regulation;

[0020] an index calculation module configured to obtain a target visual-spatial attention index of the target patient based on the target eye movement information and a preset corresponding relationship between eye movement information and the visual-spatial attention index, and obtain a target emotion index of the target patient based on the target facial video information by using a pre-trained convolutional neural network;

[0021] a parameter adjustment module configured to adjust the target transcranial light stimulation parameter based on the target brain function feedback information, the target visual-spatial attention index, and the target emotion index of the target patient;

[0022] a light emitting unit configured to adjust a light emitting parameter according to the target transcranial light stimulation parameter, the light emitting parameter comprising one or more of a wavelength, a frequency, a pulse width, and an emitting time of the light.

[0023] According to another aspect of the present application, there is provided a non-transitory computer readable storage medium having stored computer instructions for causing a computer to perform the transcranial light stimulation regulation method as described above.

[0024] The one or more technical solutions provided in the embodiments of the present application train a target multi-classifier using existing patient data, and use the target multi-classifier to output a target transcranial light stimulation parameter based on a plurality of sets of data of a target patient, and then use the target transcranial light stimulation parameter to regulate the target patient through transcranial light stimulation, thereby improving the efficiency and accuracy of obtaining the target transcranial light stimulation parameter, and improving the regulation efficiency. Furthermore, by collecting brain function, eye movement, and facial video information of the target patient during the regulation process, and adjusting the target transcranial light stimulation parameter based on the information, the transcranial light can respond to the feedback of the patient on the transcranial light stimulation parameter in a timely manner, thereby adjusting the parameters of the light emitter in a timely and accurate manner, and further improving the effect of transcranial light neural stimulation regulation. BRIEF DESCRIPTION OF DRAWINGS

[0025] In the following description of exemplary embodiments in conjunction with the accompanying drawings, more details, features and advantages of the present application are disclosed, in which:

[0026] Figure 1 A flowchart of a transcranial light stimulation regulation method provided by an embodiment of the present application;

[0027] Figure 2 A flowchart of a process of obtaining a target transcranial light stimulation parameter in a transcranial light stimulation regulation method provided by an embodiment of the present application;

[0028] Figure 3 A logical structure diagram of a transcranial light stimulation regulation system provided by an embodiment of the present application;

[0029] Figure 4 Another logical structure diagram of a transcranial light stimulation regulation system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0030] Embodiments of the present application will be described below in more detail with reference to the accompanying drawings. While several embodiments of the application are shown in the drawings, it is understood that the application can be embodied in various forms and should not be interpreted in a limited sense. Rather, these embodiments are provided so that this disclosure will be thorough and complete. It is understood that the drawings and embodiments are for illustrative purposes only and should not be construed as limiting the scope of the present application.

[0031] It should be understood that each of the steps in the method embodiments of the present application can be performed in a different order and / or in parallel. In addition, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present application is not limited in this respect.

[0032] The term "comprising" and variations thereof as used herein are used inclusively, i.e., "comprising but not limited to." The term "based on" means "based at least in part on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments." Related definitions are given below. It should be noted that the concepts mentioned in the present application are merely illustrative and not restrictive, and those skilled in the art should understand that "one", "multiple" modification is illustrative and not restrictive, unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0033] It should be noted that the "one", "multiple" modification mentioned in the present application is illustrative and not restrictive, and those skilled in the art should understand that unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0034] The names of the messages or information exchanged between the devices in the embodiments of the present application are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0035] In order to improve the efficiency of patient rehabilitation training, the present application provides a transcranial light stimulation regulation method, system and storage medium, which will be described below with reference to the accompanying drawings:

[0036] Figure 1 A flowchart of the transcranial light stimulation regulation method provided by the embodiments of the present application can include the following steps:

[0037] S101, obtain historical sample data of multiple groups of brain dysfunction patients in a history record and sample labels of the historical sample data, to obtain a target training set; wherein, each group of the historical sample data comprises near-infrared brain function data, eye movement data and facial video data; the sample label is an actual transcranial light stimulation parameter corresponding to the historical sample data, and the actual transcranial light stimulation parameter at least comprises a stimulation wavelength, a stimulation frequency, a stimulation pulse width and a stimulation time;

[0038] S102, obtain multiple groups of target patient data of a target patient to be subjected to transcranial light stimulation, to obtain a target patient data set, each group of the target patient data comprising near-infrared brain function data, eye movement data and facial video data of the target patient;

[0039] S103, train a target multi-classifier based on each piece of historical sample data in the target training set and the sample label, the target multi-classifier being used to output a transcranial light stimulation parameter;

[0040] S104, input the target patient data set to the target multi-classifier, so that the target multi-classifier outputs a target transcranial light stimulation parameter of the target patient based on the target patient data set;

[0041] S105, transcranial light stimulation is performed on the target patient by using the target transcranial light stimulation parameter, and target brain function feedback information, target eye movement information and target facial video information of the target patient are collected during the transcranial light stimulation; wherein, the target brain function feedback information is used to represent changes in brain function data of the target patient before and after transcranial light stimulation;

[0042] S106, based on the target eye movement information and a corresponding relationship between preset eye movement information and a visual spatial attention index, a target visual spatial attention index of the target patient is obtained; a target emotion index of the target patient is obtained based on the target facial video information by using a pre-trained convolutional neural network;

[0043] S107, the target transcranial light stimulation parameter is adjusted based on the target brain function feedback information, the target visual spatial attention index and the target emotion index of the target patient.

[0044] S108, adjust light emission parameters of a light emission unit according to the target transcranial light stimulation parameter, wherein the light emission parameters comprise a wavelength, a frequency, a pulse width and an emission time of light.

[0045] By applying the embodiment of the present application, the target multi-classifier is trained by using the existing patient data, and the target multi-classifier is used to output the target transcranial light stimulation parameter based on the multiple sets of data of the target patient, and then the target transcranial light stimulation parameter is used for light stimulation of the target patient, thereby improving the acquisition efficiency and accuracy of the target transcranial light stimulation parameter, improving the efficiency, and further, by collecting the brain function, eye movement and facial video information of the target patient in the stimulation process, and based on the information, the target transcranial light stimulation parameter is feedback adjusted, so that the transcranial light can respond to the feedback of the patient on the transcranial light stimulation parameter in time, thereby improving the transcranial light stimulation effect.

[0046] The above S101-S108 are exemplarily described as follows:

[0047] In a possible embodiment, the preset index data of the brain dysfunction patient and the corresponding transcranial light stimulation parameter in the process of transcranial light stimulation can be recorded with the consent of the patient, and the preset index data and the transcranial light stimulation parameter are stored in the database. Exemplarily, in the stimulation process, the preset index data of the patient and the corresponding transcranial light stimulation parameter can be collected according to a preset collection period, and the collection period can be set according to the actual application scene. Exemplarily, the near-infrared brain function parameter can be collected every 15 minutes or 20 minutes, and the collection lasts for 15 minutes or 20 minutes. The eye movement data and the facial video data can be continuously collected in the process of light stimulation regulation. For example, the light stimulation lasts for 20 minutes, and the eye movement data and the facial video data of the patient can be continuously collected in the 20 minutes.

[0048] The above preset index can include the near-infrared brain function data, eye movement data and facial video data of the patient, wherein the near-infrared brain function data refers to the physiological and functional information about the brain activity obtained by the near-infrared spectroscopy (NIRS) technology. This technology uses the characteristics of near-infrared light, which can penetrate the scalp and skull, and detect the blood dynamics changes of the cerebral cortex. The eye movement data refers to the visual behavior data recorded and analyzed by the eye tracking technology, which can include the fixation point, fixation time and frequency, saccade distance and pupil size, etc.

[0049] In actual application scenarios, a virtual reality (VR) rehabilitation training technology can be used to regulate a patient through transcranial light stimulation. The virtual reality rehabilitation training technology enables the patient to interact with a 3D visual or other sensory environment by using electronic devices for modeling and simulation. As a possible implementation, during the virtual reality rehabilitation training, a brain function imaging device, a near-infrared spectroscopy device, a camera, and an eye tracker can be accessed to collect preset index data of the patient. Specifically, the brain function imaging device and the near-infrared spectroscopy device can be used to collect near-infrared brain function data of the patient, the eye tracker can be used to collect eye movement data of the patient, and a depth camera can be used to collect facial video data of the patient.

[0050] In a possible embodiment, different groups of preset index data of the patient and corresponding transcranial light stimulation parameters can be distinguished by using a collection timestamp as a reference. For example, a timestamp label can be added to the collected data when the preset index data and the transcranial light stimulation parameters are collected. The preset index data and the transcranial light stimulation parameters with the same timestamp label are stored in the database as a group of historical data.

[0051] In S101, all data in the database can be used to construct the target training set, or part of the data in the database can be used to construct the target training set, which is not limited in the present application. The target training set includes multiple groups of historical sample data of different patients and corresponding sample labels. The historical sample data is the preset index data, and the sample label is the transcranial light stimulation parameter. For example, the transcranial light stimulation parameter can include stimulation wavelength, stimulation frequency, stimulation pulse width, and stimulation time.

[0052] In S102, multiple groups of near-infrared brain function information, eye movement information, and facial video information of the target patient can be collected through the above data collection process to obtain a target patient data set, which is not described herein.

[0053] In a possible embodiment, after obtaining the target training set, a multi-classifier can be trained based on the target training set. The multi-classifier can be constructed in any feasible structure. For example, the classifier can include an input layer, a hidden layer, and an output layer. The input layer is used to receive the input of the data, the hidden layer is used to extract the data features, and the input layer is used to output the results based on the extracted data features. As a possible implementation, the output layer of the multi-classifier can include multiple output units, each of which is used to output a different type of transcranial light stimulation parameter. For example, when the transcranial light stimulation parameter includes stimulation wavelength, stimulation frequency, stimulation pulse width, and stimulation time, the number of output units can be 4, and the four output units are used to output the stimulation wavelength, the stimulation frequency, the stimulation pulse width, and the stimulation time, respectively.

[0054] The multi-classifier can be trained in the application by any feasible method. Exemplarily, each piece of historical sample data in a target training set can be output to the multi-classifier to be trained, and a transcranial light stimulation prediction parameter output by the multi-classifier to be trained can be obtained, the light stimulation prediction parameter including a predicted wavelength, a predicted frequency, a predicted pulse width, and a predicted time. The multi-classifier is adjusted in parameters based on a difference between the preset parameter and a sample label corresponding to the historical sample data, until the difference converges, and a target multi-classifier is obtained.

[0055] After the target multi-classifier is obtained, a target patient data set can be input to the target multi-classifier, and a light stimulation parameter prediction value output by the target multi-classifier based on the target patient data set can be obtained as a target light stimulation parameter.

[0056] In a possible embodiment, in order to improve the accuracy of the multi-classifier and further improve the prediction accuracy of the target light stimulation parameter, the target training set and the target patient data set can be obtained by the following steps:

[0057] S111, a first source data set and a first target data set are obtained, the first source data set including a plurality of groups of historical sample data and a sample label of each group of historical sample data, and the first target data set including a plurality of groups of target patient data of the target patient.

[0058] The historical sample data and the corresponding sample label obtained from the database are included in the first source data set, and the first target data set includes a plurality of groups of near-infrared brain function data, eye movement data, and facial video data of the target patient collected according to a preset collection period.

[0059] S112, a first similarity of each group of historical sample data and the first target data set is calculated respectively, and historical sample data with a first similarity higher than a first preset threshold is deleted from the first source data set to obtain a second source data set.

[0060] The historical sample data and the data of the target patient both include a plurality of data types, which can include the above-mentioned near-infrared brain function data, eye movement data, and facial video data, etc. As a possible implementation, the similarity of the first source data set and the first target data set can be calculated based on the above-mentioned data types. The similarity calculation can be performed in any feasible manner. Exemplarily, the cosine similarity distance, the Mahalanobis similarity distance, etc. between each type of data can be calculated to calculate the similarity between the sample data and the target data.

[0061] Exemplarily, the similarity of each group of data in the first source data set and the first target data set can be calculated by the following formula:

[0062] A(S1, D1) = (S 1nty -D 1nty ) 2 +(S 1yty -D 1yty ) 2 +(S 1mty -D 1mty ) 2

[0063] wherein S1 is a first source data set, D1 is a first target data set, S 1nty， S 1yty， S 1mty are near-infrared brain function information data, eye movement information data, and facial video information data in historical sample data in the first source data set respectively, D 1nty， D 1yty， D 1mty are near-infrared brain function information data, eye movement information data, and facial video information data in each piece of patient data in the first target data set respectively, and A(S1, D1) represents a similarity distance between a group of historical sample data in the first source data set and target patient data in the first target data set.

[0064] For example, the first source data set includes historical sample data A, B, and C, and the first target data set includes target patient data a, b, and c, and a statistical value of the similarity between A and a, b, and c can be calculated as the similarity between A and the first target data set. The statistical value can be an average value, a median value, or the like.

[0065] In another possible embodiment, the similarity between each group of historical sample data in the first source data set and each group of target patient data in the first target data set can also be calculated respectively.

[0066] When the similarity distance is higher than a first preset threshold, it indicates that the historical sample data in the first source data set is quite different from the target patient data in the first target data set, and therefore, the historical sample data can be removed from the first source data set. Each historical sample data with a similarity distance greater than the first preset threshold between the first source data set and the first target data set is removed from the first source data set, thereby obtaining a second source data set.

[0067] S113, classifying the second source data set by using a first target classifier to obtain classification results of each group of second sample data included in the second source data set, and removing second sample data with incorrect classification results from the second source data set to obtain a third source data set.

[0068] The first target classifier can be a support vector machine (SVM), a decision tree (DT), a convolutional neural network (CNN), a long short-term memory network (LSTM), or the like, and the specific structure can be selected according to an actual application scenario.

[0069] In a possible embodiment, the first target classifier can be trained based on the second historical sample data and the sample labels. For example, the first target classifier can be selected by the following steps:

[0070] In S1131, the candidate classifiers are tested based on the second sample data in the second source data set, and the candidate classifiers with a classification accuracy higher than a preset accuracy threshold are obtained as the pending classifiers based on the sample labels corresponding to the second sample data in the second source data set. The number of the pending classifiers is preset.

[0071] In this step, the second source data set can be used to train the candidate classifiers, which can include an SVM, a DT, a CNN, an LSTM, or the like. As a possible implementation, the second sample data included in the second data set can be input into the candidate classifiers, and the classification prediction results output by the candidate classifiers for the second sample data can be obtained. The candidate classifiers can be trained based on the difference between the classification prediction results and the sample labels of the second sample data until the difference converges. The second sample data can be input into the trained candidate classifiers, and the prediction accuracy of the candidate classifiers can be determined based on the proportion of the correct classification prediction results output by the candidate classifiers for the second sample data in the number of the second sample data. Then, the candidate classifiers with the highest accuracy can be selected as the pending classifiers. The preset number can be selected according to an actual application scenario, for example, the preset number can be 4.

[0072] For example, after the prediction accuracy of each candidate classifier is obtained, the candidate classifiers can be sorted in descending order of the prediction accuracy, and the candidate classifiers ranked in the top four can be selected as the pending classifiers.

[0073] In a possible embodiment, if the candidate classifiers with the same prediction accuracy appear, resulting in a number of classifiers with the highest accuracy greater than the preset number, the pending classifiers can be randomly selected from the candidate classifiers with the same prediction accuracy, so that the number of the pending classifiers meets the preset number.

[0074] In a possible embodiment, each candidate classifier can be trained by using each historical sample data stored in the database and the sample label of the historical sample data, and each candidate classifier can be tested by using each second sample data in the second source data set to test the prediction accuracy of each candidate classifier, and then a plurality of candidate classifiers can be selected as the pending classifiers; the number of the pending classifiers is preset.

[0075] Then, the first target classifier can be determined based on the prediction result accuracy output by each of the pending classifiers for each target patient data included in the first target data set. Specifically, the process of determining the first target classifier can further include the following steps:

[0076] S1132, each of the pending classifiers is used to classify each group of target patient data in the first target data set, to obtain a plurality of candidate classification labels corresponding to the first target data set. In the case of repetition of each candidate classification label, the repeated candidate classification label is used as the target label classification result of the first target data set; wherein the target label classification result is the predicted light stimulation parameter for the first target data set.

[0077] As described above, there are a plurality of pending classifiers, and the prediction classification result output by each of the pending classifiers for each patient data in the first target data set can be the same or different. The classification result is the light stimulation parameter. In this step, if the same prediction classification result appears, the prediction classification result can be retained as the target label classification result corresponding to the corresponding patient data. The above-mentioned same can refer to two or more than two same candidate classification labels in each candidate classification label. For example, the number of pending classifiers is 4, and the candidate classification labels output by the four pending classifiers for the same group of patient data are A, A, B and C, respectively. Therefore, the prediction classification result A can be used as the target label classification result corresponding to the group of patient data.

[0078] S1133, each of the pending classifiers is used to classify each group of target patient data in the first target data set, to obtain a pending classification result.

[0079] S1134, the classification accuracy of each of the pending classifiers is determined based on the pending classification result and the target label classification result, and the pending classifier with the highest classification accuracy is selected as the first target classifier

[0080] In a possible embodiment, each target patient data in the first target data set can be input into each to-be-determined classifier, and a classification result prediction value output by each to-be-determined classifier for the patient data is obtained as a to-be-determined classification result. Whether the to-be-determined classification result is correct is determined based on a target label classification result corresponding to the patient data, and then an accuracy rate of each to-be-determined classifier for the first target data set is obtained, so that the to-be-determined classifier with the highest accuracy rate is selected as the first target classifier.

[0081] By using the second source data set and the first target classifier to test the candidate classifier multiple times, and then selecting the candidate classifier with the best performance in the two data sets, that is, the candidate classifier with the highest prediction accuracy, as the first target classifier, the accuracy of the classification result output by the second source data set can be improved.

[0082] In a possible embodiment, each second sample data included in the second source data set can be input into the first target classifier, and a classification result output by the first target classifier for each second sample data is obtained, which is a light stimulation parameter prediction value output by the first target classifier for the second sample data. Correspondingly, if the classification result corresponding to the second sample data is inconsistent with the sample label corresponding to the second sample, it indicates that the second sample data or the sample label thereof is abnormal, and therefore, the second sample label can be excluded from the second source data set, so as to obtain a third source data set.

[0083] In a possible embodiment, a similarity distance between each third sample data in the third source data set and each target patient data in the first target data set can be calculated, and third samples and target patient data with a similarity distance greater than a second preset threshold value are deleted from the third source data set and the first target data set to obtain a target training set and a target patient data set. The second preset threshold value can be higher than the first preset threshold value.

[0084] As a possible implementation, the process of obtaining the target training set and the target patient data set can further include the following steps:

[0085] S114, each group of third sample data in the third source data set is input into a plurality of preset binary classifiers together with each group of target sample data in the first target data set, and a loss function value output by each preset classifier is obtained; the loss function in each preset binary classifier contains different data types, and the data types include near-infrared brain function data, eye movement data and facial video data; the number of preset binary classifiers is the same as the number of data types included in the sample data.

[0086] In this step, a set of third sample data and a set of target patient data can be input into each preset binary classifier at the same time, and the preset binary classifier can output whether the third sample data and the target patient data are similar based on the similarity of the third sample data and the target patient data. The above-mentioned plurality of preset binary classifiers can be selected according to the actual application scene, such as the above-mentioned SVM, DT, CNN, LSTM and the like. The above-mentioned plurality of preset binary classifiers can be the same type of classifier or different types of classifiers, and the present application does not make specific limitation thereto.

[0087] The loss functions used by each preset binary classifier are different. Specifically, the loss functions used by each classifier can be obtained based on different types of historical sample data and patient data. For example, each loss function can be obtained based on the cross-entropy loss function of near-infrared brain function information, eye movement information and facial video information.

[0088] As a possible implementation, the loss functions of the above-mentioned plurality of preset binary classifiers can be the following functions respectively:

[0089]

[0090] wherein G(f nsy ) is the cross-entropy loss function of the near-infrared brain function information data binary classifier f nsy , n and m are respectively the number of third sample data in the third source data set and the number of target patient data in the first target data set, S 3nsyi is the i-th near-infrared brain function sample data in the third source data set, D 1nsyj is the j-th near-infrared brain function data in the first target data set, and a is a preset first correction coefficient.

[0091]

[0092] wherein G(f ysy ) is the cross-entropy loss function of the eye movement information data binary classifier f ysy , n and m are respectively the number of third sample data in the third source data set and the number of target patient data in the first target data set, S 3ysyi is the i-th eye movement sample data in the third source data set, D 1nsyj is the j-th eye movement data in the first target data set, and β is a preset second correction coefficient.

[0093]

[0094] wherein G(f ysy ) is the cross-entropy loss function of the facial video information data binary classifier f msycross-entropy loss function, n and m are the number of third sample data in the third source data set and the number of target patient data in the first target data set, S 3msyi is the i-th face video sample data in the third source data set, D 1msyj is the j-th face video data in the first target data set, and γ is a preset third correction coefficient.

[0095] S115, in the case where each of the loss function values reaches a minimum value, determining the target data type corresponding to the preset binary classifier with the lowest accuracy among the preset binary classifiers.

[0096] In this step, the preset binary classifier with the largest difference between the similarity results output by other binary classifiers can be determined as the preset binary classifier with the lowest accuracy based on the similarity results between the third sample data and the target patient data output by each preset binary classifier. In another possible embodiment, the preset binary classifier with the lowest accuracy can also be determined based on the above similarity results and the actual similarity results between the sample labels corresponding to the third sample data and the target classification labels corresponding to the target patient data.

[0097] As described above, different preset binary classifiers set cross-entropy loss functions based on different types of data, so in this step, the data type included in the cross-entropy loss function used by the preset binary classifier with the lowest accuracy can be determined as the target data type.

[0098] S116, calculating the similarity distance between the target data type in the third source data set and the target data type in the first target data set, and deleting the third sample data and the target patient data with a similarity distance greater than a second preset threshold from the third source data set and the first target data set, to obtain a fourth source data set and a second target data set, and taking the fourth source data set as a target training set and the second target data set as a target patient data set.

[0099] For example, when the above three loss functions all reach a minimum value, the accuracy of the binary classifier corresponding to the near-infrared brain function information data is the lowest, then the similarity distance between the near-infrared brain function data in the third sample data in the third source data set and the near-infrared brain function information data in the target patient data can be calculated, and if the similarity distance is greater than a preset distance threshold, the third sample data can be deleted, and the corresponding target patient data can be deleted from the first target data set to obtain a second target data set.

[0100] The original source data set and the target patient data set are filtered through the technical means, to obtain a target training set and a target data set, so that the data similarity between the target training set and the target data set is high, and then when the multi-classifier trained in the target training set is used to predict the parameters of the target data set, better results can be achieved, and the accuracy of the obtained target transcranial light stimulation parameters is further improved.

[0101] As shown in Figure 2 , Figure 2 is a flowchart of obtaining light stimulation parameters in an embodiment of the present application. Specifically, first, a first source data set and a first target data set are obtained. Based on the similarity between the data in the first source data set and the first target data set, sample data with a similarity lower than a first preset threshold is removed from the first source data set to obtain a second data set. The first target classifier is selected using the second source data set and the first target data set, and the classification results of each second sample data in the second source data set are calculated using the first target classifier. The second sample data with incorrect classification results is removed from the second source data set to obtain a third source data set. The similarity results between the third source data set and the first target data set are calculated using a plurality of preset binary classifiers. Based on the similarity results and the data types included in the cross-entropy loss function used by each preset binary classifier, the target data type to be calculated is determined. The similarity between the data corresponding to the target data type in the third source data set and the first target data set is calculated. The third sample data and the target patient data with a similarity lower than a second preset threshold are removed from the third source data set and the first target data set, to obtain a fourth source data set and a second target data set, which are used as a target training set and a target patient data set, respectively. The multi-classifier is trained using the target training set, and the light stimulation parameters predicted for the target patient based on the second target data set are output using the classifier.

[0102] After obtaining the light stimulation parameters, the light stimulation parameters can be used to control the transcranial light to stimulate the target patient in the virtual reality rehabilitation training task. In one possible embodiment, the light emission parameters of the light emission unit can be adjusted according to the light stimulation parameters, so that the light emission unit emits light that meets the above-mentioned target transcranial light stimulation parameters. The light emission parameters can include the wavelength, frequency, pulse width and emission time of the emitted light. As one possible implementation, the wavelength of the emitted light can be adjusted according to the stimulation wavelength included in the target transcranial light stimulation parameters, the frequency of the emitted light can be adjusted according to the stimulation frequency included in the target transcranial light stimulation parameters, the pulse width of the emitted light can be adjusted according to the stimulation pulse width included in the target transcranial light stimulation parameters, and the time length of the emitted light can be adjusted according to the stimulation time length included in the target transcranial light stimulation parameters.

[0103] In a possible embodiment, the brain function feedback information of the target patient, the eye movement information and the facial video information of the target patient can be collected during the transcranial light stimulation. The collection of the eye movement information and the facial video information has been described in detail above, and will not be repeated here.

[0104] The target brain function feedback information of the target patient is used to represent the change of the brain function of the target patient before and after the transcranial light stimulation. In a possible embodiment, the target brain function feedback information includes a target brain activation value index and a target brain function network connection index. The target brain activation value index is obtained by the following formula:

[0105]

[0106] wherein GJ rht is the target brain activation value index; m, n, and l are respectively the number of the brain frontal lobe, the brain occipital lobe, and the brain parietal lobe blood oxygen channels; W qei、 W hei is the wavelet amplitude of the brain blood oxygen signal of the i th channel of the brain frontal lobe before and after the light stimulation; W qzi、 W hzi is the wavelet amplitude of the brain blood oxygen signal of the i th channel of the brain occipital lobe before and after the light stimulation; W qdi、 W hdi is the wavelet amplitude of the brain blood oxygen signal of the i th channel of the brain parietal lobe before and after the light stimulation; W qxi、 W hxi is the wavelet amplitude of the brain blood oxygen signal of the i th channel of the brain frontal lobe, the brain occipital lobe, and the brain parietal lobe before and after the light stimulation; and λ is a preset fourth correction coefficient, which can be any constant set according to the actual application scenario.

[0107] The target brain function network connection index is obtained by the following formula:

[0108]

[0109] wherein BM is the target brain function network connection index, Le, Lz, and Ld are respectively the number of the channels with functional connection in the brain frontal lobe, the brain occipital lobe, and the brain parietal lobe, φ Lehi , φ Leqi is the connection efficiency of the i th channel with functional connection in the brain frontal lobe before and after the light stimulation, φ Lzhi , φ Lzqi is the connection efficiency of the i th channel with functional connection in the brain occipital lobe before and after the light stimulation, φ Ldhi , φ Ldqi is the connection efficiency of the i th channel with functional connection in the brain parietal lobe before and after the light stimulation, and α1, α2, and α3 are respectively a preset fifth correction coefficient, a preset sixth correction coefficient, and a preset seventh correction coefficient.

[0110] In a possible embodiment, after obtaining the target eye movement information, the target visual-spatial attention of the target patient can be obtained based on the target eye movement information and a preset corresponding relationship between eye movement information and a visual-spatial attention index. Specifically, the target visual-spatial attention SZ can be calculated based on the target eye movement information by the following formula: y

[0111]

[0112] wherein HY and LY are high-frequency power and low-frequency power of the eye movement signal respectively, P k (e jw ) is a power spectrum of the eye movement signal in different sampling periods, σ(e jw ) is a power spectrum of the impulse function, ZX t is a positive saccade delay time of the time-domain feature of the eye movement signal, FX t is a negative saccade delay time of the time-domain feature of the eye movement signal, and α, β, and γ are weight coefficients, e is a natural constant, w is a center frequency of the eye movement signal, and j is an imaginary part of a complex number.

[0113] Based on a preset corresponding relationship between a visual-spatial attention range and a visual-spatial attention index, the target visual-spatial attention index of the target patient is obtained based on the target visual-spatial attention.

[0114] For example, the value range of SZ y is set to be between 0 and 1, wherein (0-0.4) indicates that the visual-spatial attention index is better, [0.4-0.7) indicates that the visual-spatial attention index is general, and [0.7-1] indicates that the visual-spatial attention index is relatively poor.

[0115] In a possible embodiment, the target emotional index of the target patient is obtained based on the target facial video information by using a pre-trained convolutional neural network, which includes:

[0116] Each target facial video segment is obtained by cropping the target facial video, and a convolutional neural network is used to output a plurality of target emotional states of the target patient based on each target facial video segment, wherein the target emotional states include negative, positive, and neutral. Based on a preset proportion of a preset emotional state in the target emotional states and a preset corresponding relationship between a proportion range and an emotional index, the target emotional index of the target patient is determined.

[0117] ​In this step, the target face video can be cropped based on a preset segment length to obtain a plurality of target video segments. In one possible embodiment, different segment lengths can be set for cognitive tasks of different difficulty coefficients, and the preset segment length can be determined based on the difficulty coefficient of the cognitive task set for the target patient, and then the target face video can be cropped based on the segment length to obtain a plurality of target video segments.

[0118] Then, the pre-trained convolutional neural network can be used to output the target emotional state of the target patient based on each target face video segment. The convolutional neural network can be pre-trained based on a large number of face video segments and the emotional state corresponding to the face video segments. The target emotional state can include negative, positive, and neutral.

[0119] For example, the acquisition process of the target emotional state can be represented by the following formula:

[0120] QX i =G CNN (W CNN ,B CNN ;JJ CNN ,CH CNN ;SP i ,HK i )

[0121] Wherein, W CNN is the weight matrix of the convolutional neural network model, B CNN is the bias parameter; JJ CNN is the convolution layer of the convolution module, and the convolution kernel is 3*2 and the convolution step is 2*2; CH CNN is the pooling layer of the convolution module, and the pooling layer is a 2*2 maximum pooling kernel; SP i is the i-th cropped face video; SP i is the difficulty coefficient of the virtual reality task corresponding to the i-th face video; QX i is the emotional state corresponding to the i-th face video, including negative, positive, and neutral.

[0122] Then, the target request index can be calculated based on each target emotional state. Specifically, the proportion of the target emotional state in each emotional state can be calculated, and the target emotional index can be obtained based on the corresponding relationship between the data range of the proportion and the preset emotional index. For example, the target emotional index of the target patient can be calculated by the following formula:

[0123]

[0124] Wherein, SL cj is the number of positive emotions, SL czThe number of neutral emotions is N, and N is the total number of emotional states; LQ is an emotional index. The LQ value ranges from 0 to 1, wherein [0.7-1) indicates that the emotional index is excellent, [0.5-0.7) indicates that the emotional index is good, [0.3-0.5] indicates that the emotional index is poor, and (0-0.3) indicates that the emotional index is extremely poor.

[0125] In a possible embodiment, the light stimulation parameters can be adjusted based on the target brain function feedback information collected during the transcranial light stimulation process, the target visual spatial attention index, the target emotional index, and a preset adjustment strategy, which can be set according to an actual application scenario.

[0126] In a possible embodiment, the stimulation duration in the target transcranial light stimulation parameter can be adjusted based on the target visual spatial attention index, the stimulation frequency in the target transcranial light stimulation parameter can be adjusted based on the target emotional index, the stimulation wavelength in the target transcranial light stimulation parameter can be adjusted based on the target brain activation index, and the stimulation pulse width in the target transcranial light stimulation parameter can be adjusted based on the target brain network connection index.

[0127] For example, when the target visual spatial attention index is good, the stimulation time is lengthened; when the target visual spatial attention index is general, the stimulation time remains unchanged; and when the target visual spatial attention index is poor, the stimulation time is shortened. When the target emotional index is excellent, the stimulation frequency is increased; when the target emotional index is good, the stimulation frequency remains unchanged; when the target emotional index is poor and very poor, the stimulation frequency is decreased. When the brain activation value index output by the brain function evaluation module exceeds a threshold M1, the light stimulation wavelength is decreased; when the target brain activation value index does not exceed the threshold M1, the light stimulation wavelength is increased; when the target brain network connection index exceeds a threshold M2, the light stimulation pulse width is decreased; and when the target brain network connection index does not exceed the threshold M2, the light stimulation pulse width is increased.

[0128] In a possible embodiment, after the target transcranial light stimulation parameter is adjusted based on the feedback information of the target patient, the parameter of the light emission unit can be adjusted based on the adjusted target transcranial light stimulation parameter, so that the light emission unit can timely and accurately emit transcranial light according to the adjusted target transcranial light stimulation parameter.

[0129] In a possible embodiment, the control parameters used by the target patient in each transcranial light stimulation can be stored in correspondence with the target patient information and used as the initial value of the control parameters for the next transcranial light stimulation of the target patient.

[0130] In a possible embodiment, the above-mentioned light stimulation parameters, target brain function feedback information, target visual space indicators and the like can be displayed through a display device to present the transcranial light stimulation effect and the light stimulation parameter adjustment to relevant personnel in real time.

[0131] By applying the embodiment of the present application, the cognitive regulation state of the patient is evaluated through the visual space attention indicators, emotion indicators, brain activation indicators and brain network connection indicators, and the synergistic control effect of the multi-source information such as the brain and physiological information before, during and after the virtual training process of the patient is fully considered, so that the brain function state of the patient can be comprehensively evaluated.

[0132] Moreover, the transcranial light stimulation parameters of the patient are adjusted in real time by fully utilizing the multi-level information changes of the visual space attention indicators, emotion indicators, brain activation indicators and brain network connection indicators of the patient, so as to promote the synergistic optimization and real-time feedback of the brain and the light stimulation data. The patient can be provided with an individualized adaptive light stimulation parameter adjustment scheme, so that the light stimulation assisted rehabilitation training can play the maximum gain effect, and the rehabilitation training efficiency and effect of the patient can be improved.

[0133] Based on the same inventive concept, the embodiment of the present application also provides a transcranial light stimulation regulation system, as shown in Figure 3 The system 300 can include:

[0134] A first acquisition module 301 is configured to acquire historical sample data of a plurality of groups of brain dysfunction patients in historical records and sample labels of the historical sample data, to obtain a target training set. Each group of the historical sample data includes near-infrared brain function data, eye movement data and facial video data. The sample label is an actual transcranial light stimulation parameter corresponding to the historical sample data, and the actual transcranial light stimulation parameter at least includes a stimulation wavelength, a stimulation frequency, a stimulation pulse width and a stimulation time.

[0135] A second acquisition module 302 is configured to acquire a plurality of groups of target patient data of a target patient to be subjected to transcranial light stimulation, to obtain a target patient data set. Each group of the target patient data includes near-infrared brain function data, eye movement data and facial video data of the target patient.

[0136] A classifier training module 303 is configured to train a target multi-classifier based on each piece of the historical sample data and the sample label in the target training set. The target multi-classifier is configured to output a transcranial light stimulation parameter.

[0137] A parameter output module 304 is configured to input the target patient data set to the target multi-classifier, so that the target multi-classifier outputs a target transcranial light stimulation parameter of the target patient based on the target patient data set.

[0138] The information collection module 305 is configured to perform transcranial light neural regulation on the target patient by using the target transcranial light stimulation parameter, and collect target brain function feedback information, target eye movement information and target facial video information of the target patient during the transcranial light stimulation, wherein the target brain function feedback information is used to represent the change of the brain function data of the target patient before and after the transcranial light stimulation.

[0139] The index calculation module 306 is configured to obtain a target visual spatial attention index of the target patient based on the target eye movement information and a preset corresponding relationship between eye movement information and the visual spatial attention index, and obtain a target emotion index of the target patient based on the target facial video information by using a pre-trained convolutional neural network.

[0140] The parameter adjustment module 307 is configured to adjust the target transcranial light stimulation parameter based on the target brain function feedback information, the target visual spatial attention index and the target emotion index of the target patient.

[0141] The light emitting unit 308 is configured to adjust the light emission parameter of the light emitting unit according to the target transcranial light stimulation parameter emitted by the parameter adjustment module 307, including the wavelength, frequency, pulse width and emission time of the light.

[0142] In a possible embodiment, as shown in Figure 4 The transcranial light stimulation regulation system can include a cranium light field model module, a virtual reality task module, an information collection and analysis module, a light stimulation module, a light emission control module, a brain function evaluation module and a feedback module.

[0143] The cranium light field model module is configured to construct a target multi-classifier, and output transcranial light stimulation parameter information based on a target patient data set by using the target multi-classifier. The virtual reality module is configured to provide a virtual reality rehabilitation training task for a user. The light emission control module is configured to adjust the parameter of the light stimulation module according to the transcranial light stimulation parameter output by the cranium light field model module, and the light stimulation module is configured to perform light stimulation on the patient. The information collection and analysis module is configured to collect and analyze the eye movement and facial video information during the virtual reality rehabilitation training task of the user during the transcranial light stimulation. The brain function evaluation module is configured to evaluate the brain function of the patient before and after the transcranial light regulation. The feedback module is configured to feed back the eye movement and facial video information output by the information collection and analysis module and the brain function information output by the brain function evaluation module to the light emission control module for adjustment of the light stimulation parameter.

[0144] In the present application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information comply with relevant laws and regulations, and do not violate public order and good customs.

[0145] The exemplary embodiments of this application further provide a non-transitory computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, causes the computer to perform the method according to the embodiments of the application.

[0146] The exemplary embodiments of this application further provide a computer program product comprising a computer program, wherein the computer program, when executed by a processor of a computer, causes the computer to perform the method according to the embodiments of the application.

[0147] The program code for implementing the method of the application can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, causes the machine to perform the functions / acts specified in the flow diagrams and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, and partially on a remote machine or entirely on a remote machine or server.

[0148] In the context of the present application, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0149] As used in the present application, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal that can be used to provide machine instructions and / or data to a programmable processor.

Claims

1. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to enable the computer to acquire historical sample data of multiple groups of patients with brain dysfunction in the historical record and the sample labels of the historical sample data, so as to obtain the target training set. Multiple sets of target patient data for the target patients to be subjected to transcranial photostimulation are obtained to obtain the target patient dataset; A target multi-classifier is trained based on each historical sample data and sample label in the target training set. The target multi-classifier is used to output transcranial photostimulation parameters. The target patient dataset is input into the target multi-classifier, so that the target multi-classifier outputs the target transcranial photostimulation parameters of the target patient based on the target patient dataset; The target patient is subjected to transcranial photostimulation modulation using the target transcranial photostimulation parameters, and target brain function feedback information, target eye movement information, and target facial video information of the target patient are collected during the transcranial photostimulation modulation process. Based on the target eye movement information and the correspondence between the preset eye movement information and the visuospatial attention index, the target visuospatial attention index of the target patient is obtained. The target emotion index of the target patient is obtained based on the target facial video information using a pre-trained convolutional neural network. The target transcranial photostimulation parameters are adjusted based on the target patient's target brain function feedback information, target visuospatial attention index, and target emotion index. The light emission parameters of the light emission unit are adjusted according to the target transcranial light stimulation modulation parameters. The light emission parameters include one or more of the following: wavelength, frequency, pulse width, and emission time.

2. The non-transitory computer-readable storage medium according to claim 1, characterized in that, The target training set and the target patient dataset are obtained through the following steps: Obtain a first source dataset and a first target dataset. The first source dataset includes multiple sets of historical sample data and sample labels for each set of historical sample data. The first target dataset includes multiple sets of target patient data for the target patient. Calculate the first similarity between the multiple sets of historical sample data and the first target dataset, and delete the historical sample data with the first similarity higher than the first preset threshold from the first source dataset to obtain the second source dataset; The second source dataset is classified using a first target classifier to obtain the classification results of each group of second sample data included in the second source dataset. The second sample data with incorrect classification results are deleted from the second source dataset to obtain the third source dataset. Calculate the similarity distance between each third sample data in the third source dataset and each target patient data in the first target dataset, and delete the third samples and target patient data with similarity distances greater than a second preset threshold from the third source dataset and the first target dataset to obtain the target training set and the target patient dataset.

3. The non-transitory computer-readable storage medium according to claim 2, characterized in that, The computer instructions are also used for: Multiple candidate classifiers are tested based on each second sample data in the second source dataset, and multiple candidate classifiers with classification accuracy higher than a preset accuracy threshold are obtained based on the sample labels corresponding to each second sample data in the second source dataset, and are used as undetermined classifiers. The first target classifier is determined based on the accuracy of the prediction results output by each of the undetermined classifiers for each target patient data contained in the first target dataset.

4. The non-transitory computer-readable storage medium according to claim 1, characterized in that, The target brain function feedback information includes target brain activation indicators and target brain function network connectivity indicators. Adjusting the target transcranial optical stimulation parameters based on the target patient's target brain function feedback information, target visuospatial attention indicators, and target emotion indicators includes: The stimulation duration in the target transcranial photostimulation parameters is adjusted based on the target visuospatial attention index; the stimulation frequency in the target transcranial photostimulation parameters is adjusted based on the target emotion index; the stimulation wavelength in the target transcranial photostimulation parameters is adjusted based on the target brain activation index; and the stimulation pulse width in the target transcranial photostimulation parameters is adjusted based on the target brain functional network connectivity index.

5. A transcranial optical stimulation modulation system, characterized in that, The system includes: The first acquisition module is used to acquire historical sample data of multiple groups of patients with brain dysfunction in the historical records and the sample labels of the historical sample data to obtain the target training set. The second acquisition module is used to acquire multiple sets of target patient data of the target patient to be subjected to transcranial photostimulation modulation, and obtain the target patient dataset. The classifier training module is used to train a target multi-classifier based on each historical sample data and sample label in the target training set. The target multi-classifier is used to output transcranial photostimulation parameters. The parameter output module is used to input the target patient dataset into the target multi-classifier, so that the target multi-classifier outputs the target transcranial photostimulation parameters of the target patient based on the target patient dataset. The information acquisition module is used to perform transcranial light stimulation modulation on the target patient using the target transcranial light stimulation parameters, and to acquire the target brain function feedback information, target eye movement information and target facial video information of the target patient during the transcranial light stimulation modulation process. The index calculation module is used to obtain the target visuospatial attention index of the target patient based on the target eye movement information and the correspondence between preset eye movement information and visuospatial attention index; and to obtain the target emotion index of the target patient based on the target facial video information using a pre-trained convolutional neural network. The parameter adjustment module is used to adjust the target transcranial photostimulation parameters based on the target patient's target brain function feedback information, target visuospatial attention index, and target emotion index. The light emission unit is used to adjust the light emission parameters according to the target transcranial light stimulation parameters, wherein the light emission parameters include one or more of the following: wavelength, frequency, pulse width, and emission time.

Images