Multi-modal biofeedback closed-loop nerve regulation method and storage medium

Abstract

The invention provides a multi-modal biofeedback closed-loop nerve regulation method and a storage medium, the multi-modal biofeedback closed-loop nerve regulation method comprises the following steps: collecting brain wave data of a regulation target, and extracting electroencephalogram characteristics of the brain wave data; constructing a neural network model, and acquiring stimulation parameters according to the electroencephalogram characteristics; and based on the stimulation parameter, stimulating the ear nerve of the regulation target through an ear nerve regulation technology. According to the multi-modal biofeedback closed-loop nerve regulation and control method and the storage medium provided by the invention, priori knowledge is provided for adaptive adjustment of ear regulation and control parameters through multi-modal data fusion based on the high-precision nerve activity reference signal provided by scalp electroencephalogram, the targeting of ear nerve regulation and control is enhanced, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of brain-computer interfaces and neuromodulation, and particularly relates to a closed-loop neuromodulation method with multimodal biofeedback and a storage medium. Background Art

[0002] Scalp brain-computer interface technology is portable and does not require craniotomy. Transauricular neuromodulation technology directly regulates the activities of the autonomic nervous system and limbic brain regions in a non-invasive manner through targets such as the auricular branch of the vagus nerve in the ear, and its portability further breaks through the scenario limitations of traditional devices.

[0003] During the conception and implementation of this application, the applicant found at least the following problems: Existing transauricular modulation systems mostly rely on fixed stimulation parameters and are difficult to dynamically optimize according to individual physiological feedback, resulting in large fluctuations in the treatment effect, which restricts the realization of precise modulation and real-time closed-loop feedback. Summary of the Invention

[0004] To alleviate the above problems, the main object of this application is to propose a closed-loop neuromodulation method with multimodal biofeedback, including:

[0005] S10: Collect electroencephalogram data of a regulation target, and extract electroencephalogram features of the electroencephalogram data;

[0006] S20: Construct a neural network model, and obtain stimulation parameters according to the electroencephalogram features;

[0007] S30: Based on the stimulation parameters, stimulate the ear nerves of the regulation target through transauricular neuromodulation technology.

[0008] Optionally, the electroencephalogram features include frequency-domain features; during the process of collecting electroencephalogram data of the regulation target and extracting electroencephalogram features of the electroencephalogram data, perform Fourier transform on the electroencephalogram data to obtain the frequency-domain features of the electroencephalogram data.

[0009] Optionally, during the process of performing Fourier transform on the electroencephalogram data to obtain the frequency-domain features of the electroencephalogram data, obtain the features of the δ wave (0.5 - 4 Hz), θ wave (4 - 8 Hz), α wave (8 - 13 Hz), β wave (13 - 40 Hz), and γ wave (40 - 100 Hz) frequency bands as the frequency-domain features.

[0010] Optionally, the electroencephalogram features include time-domain features; during the process of collecting electroencephalogram data of the regulation target and extracting electroencephalogram features of the electroencephalogram data, use the Hjorth method to quantify the signal activity degree and structure of the electroencephalogram data, calculate the activity parameter and mobility parameter of the electroencephalogram data, and obtain the time-domain features of the electroencephalogram data.

[0011] Optionally, the EEG features include entropy features; in the process of collecting the EEG data of the acquisition regulation target and extracting the EEG features of the EEG data, the time series complexity of the EEG data is quantified based on approximate entropy and sample entropy to obtain the entropy features of the EEG data.

[0012] Optionally, in the process of constructing the neural network model and obtaining the stimulation parameters according to the EEG features, it includes:

[0013] Construct the EEG features of the EEG data into a training vector, and train the neural network model to establish a non-linear mapping between the EEG features and the stimulation parameters;

[0014] Based on the trained neural network model, optimize the parameters of the neural network model, and obtain the stimulation parameters according to the EEG features of the regulation target.

[0015] Optionally, the closed-loop neuromodulation method for multimodal biofeedback further includes:

[0016] S40: Extract the electromyogram data and electrocardiogram data before and after stimulation, and perform multimodal feature calculation to quantify the regulation effect.

[0017] Optionally, in the process of extracting the electromyogram data and electrocardiogram data before and after stimulation and performing multimodal feature calculation to quantify the regulation effect, it includes:

[0018] Obtain the standard deviation of consecutive R-R intervals according to the electrocardiogram data, calculate the electrocardiogram time domain index to evaluate the autonomic nerve activity of the regulation target;

[0019] Calculate the muscle activation degree according to the electromyogram data, obtain the root mean square value of the electromyogram of the target muscle group signal, quantify the improved value of the quantitative motor function of the regulation target, and obtain the electromyogram index;

[0020] According to the electrocardiogram time domain index and the electromyogram index, calculate the stimulation effect score for the regulation target based on a preset model.

[0021] Optionally, in the process of calculating the stimulation effect score for the regulation target based on the electrocardiogram time domain index and the electromyogram index according to a preset model, the calculation is performed according to the following expression:

[0022]

[0023] where, ΔHRV is the change amount of the electrocardiogram time domain index; HRV base is the baseline value of the electrocardiogram time domain index; EMG post is the electromyogram index after stimulation; EMG pre is the electromyogram index before stimulation; w1 and w2 are weight values.

[0024] The present application also provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a closed-loop neuromodulation method for multimodal biofeedback as described above are implemented.

[0025] A closed-loop neuromodulation method for multimodal biofeedback and a storage medium provided by the present application collect electroencephalogram data of a regulation target, extract electroencephalogram features of the electroencephalogram data; construct a neural network model, and obtain stimulation parameters according to the electroencephalogram features; based on the stimulation parameters, stimulate the ear nerves of the regulation target through ear neuromodulation technology; based on the high-precision nerve activity reference signal provided by scalp electroencephalogram, through multimodal data fusion, provide prior knowledge for the adaptive adjustment of the ear regulation parameters, can be dynamically optimized according to individual physiological feedback, greatly reduce the volatility of the curative effect, realize precise regulation and real-time closed-loop feedback, enhance the targeting of ear neuromodulation, and improve the user experience. Description of the Drawings

[0026] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0027] Figure 1 It is a flowchart of a closed-loop neuromodulation method for multimodal biofeedback according to an embodiment of the present application.

[0028] Figure 2 It is a schematic diagram of the construction logic of a neural network model according to an embodiment of the present application.

[0029] The realization, functional characteristics and advantages of the purpose of the present application will be further described in conjunction with the embodiments with reference to the drawings. Through the above drawings, the specific embodiments of the present application have been shown, and there will be more detailed descriptions later. These drawings and the textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed Embodiments

[0030] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0031] It should be noted that in this document, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising such element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined based on their interpretations in the specific embodiments or further in combination with the context of the specific embodiments.

[0032] It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this document, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining". Furthermore, as used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising", "including" indicate the presence of the stated features, steps, operations, elements, components, items, types, and / or groups, but do not exclude the presence, occurrence or addition of one or more other features, steps, operations, elements, components, items, types, and / or groups. The terms "or", "and / or", "including at least one of the following" and the like used in the present application can be interpreted inclusively, or mean any one or any combination. For example, "including at least one of the following: A, B, C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C", and again, "A, B or C" or "A, B and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A and B and C". An exception to this definition only occurs when the combination of elements, functions, steps or operations is inherently mutually exclusive in some way.

[0033] It should be understood that although the steps in the flowcharts in the embodiments of the present application are sequentially shown according to the indications of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limitation, and they can be executed in other orders. Moreover, at least a part of the steps in the figure may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments, and their execution order is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0034] First Embodiment

[0035] The main purpose of the present application is to propose a closed-loop neuromodulation method for multimodal biofeedback. Figure 1 It is a flowchart of a closed-loop neuromodulation method for multimodal biofeedback according to an embodiment of the present application.

[0036] As Figure 1 shown, in one embodiment, a closed-loop neuromodulation method for multimodal biofeedback includes:

[0037] S10: Collect electroencephalogram data of a regulation target, and extract electroencephalogram features of the electroencephalogram data.

[0038] Electroencephalogram (EEG) is a method of recording brain activity using electrophysiological indicators. When the brain is active, it is formed by the summation of postsynaptic potentials synchronously generated by a large number of neurons. It records the electrical wave changes during brain activity and is the overall reflection of the electrophysiological activities of brain nerve cells on the cerebral cortex or the scalp surface. Electroencephalogram originates from the postsynaptic potentials at the apical dendrites of pyramidal cells. The formation of electroencephalogram synchronous rhythm is also related to the activities of the cortical thalamic nonspecific projection system. Electroencephalogram is the basic theoretical research of brain science, and electroencephalogram monitoring is widely used in its clinical practice applications.

[0039] Thought activities can reflect the connections between neurons in the brain. Neurons in the brain receive signals from other neurons. When the energy accumulation of these signals exceeds a certain threshold, electroencephalogram will be generated. Exemplarily, in order to detect electroencephalogram, people usually place electrodes on the human scalp to detect electroencephalogram signals, and then use related devices to collect and process electroencephalogram waves.

[0040] Electroencephalogram (EEG) is a method widely used for recording and monitoring brain electrical activities. Extracting information from EEG through machine learning algorithms can help diagnose various diseases (such as epilepsy, Alzheimer's disease, and schizophrenia) and identify various brain states. EEG is a non-invasive method that can directly measure neural activities through electrodes on the scalp. The synchronous activities of a large number of neurons generate an electric field strong enough to reach the scalp, which is recorded as an electroencephalogram with high temporal resolution. Compared with other neuroimaging methods (such as fMRI and fNIRS), directly recording neural activities is one of the advantages of EEG. In addition, due to the high temporal resolution of EEG, a wide range of neural oscillations can be captured.

[0041] S20: Construct a neural network model and obtain stimulation parameters according to the brain electrical characteristics.

[0042] Neural Networks (NN) is a complex network system formed by a large number of simple processing units (called neurons) widely interconnected. It reflects many basic characteristics of the human brain function and is a highly complex non-linear dynamic learning system. Neural networks have the capabilities of large-scale parallelism, distributed storage and processing, self-organization, self-adaptation, and self-learning, and are particularly suitable for processing imprecise and fuzzy information processing problems that require considering many factors and conditions simultaneously. Exemplarily, through a neural network model, electroencephalogram data can be subjected to feature extraction and classification analysis, thereby mapping the electroencephalogram data to a specific output result.

[0043] S30: Based on the stimulation parameters, stimulate the ear nerves of the regulation target through ear nerve regulation technology.

[0044] Scalp brain-computer interface technology has the characteristics of portability and does not require craniotomy. Trans-ear nerve regulation technology directly regulates the activities of the autonomic nervous system and limbic brain regions in a non-invasive manner through targets such as the auricular branch of the vagus nerve in the ear. Its portability further breaks through the scenario limitations of traditional devices. Exemplarily, high-precision neural activity reference signals are provided through scalp electroencephalogram signals, and through multi-modal data fusion, prior knowledge is provided for the adaptive adjustment of trans-ear regulation parameters, thereby enhancing the targeting.

[0045] In this embodiment, by collecting the electroencephalogram data of the regulation target, the electroencephalogram features of the electroencephalogram data are extracted; a neural network model is constructed, and stimulation parameters are obtained according to the electroencephalogram features; based on the stimulation parameters, the ear nerves of the regulation target are stimulated through the ear nerve regulation technology; based on the high-precision nerve activity reference signal provided by scalp electroencephalogram, through multimodal data fusion, prior knowledge is provided for the adaptive adjustment of the regulation parameters through the ear, which can be dynamically optimized according to individual physiological feedback, greatly reducing the volatility of the treatment effect, realizing precise regulation and real-time closed-loop feedback, and enhancing the targeting of ear nerve regulation.

[0046] Optionally, the electroencephalogram features include frequency domain features; in the process of collecting the electroencephalogram data of the regulation target and extracting the electroencephalogram features of the electroencephalogram data, the electroencephalogram data is subjected to Fourier transform to obtain the frequency domain features of the electroencephalogram data.

[0047] Feature extraction refers to a set of methods for reducing the dimensionality of input data by measuring and extracting specific information. EEG signals contain a large amount of information about space, time, and spectrum. This makes EEG a suitable method for studying brain function and cognition. Exemplarily, for electroencephalogram (EEG) data, there are multiple methods to extract features from one or a combination of the time domain, frequency domain, or spatial domain. The fast Fourier transform (FFT) is a common way to identify the frequency components of EEG signals, and frequency domain features can be extracted from the frequency representation of EEG signals.

[0048] The frequency resolution depends on the length of the signal and is independent of the sampling frequency. One assumption of the Fourier transform is that the signal is stationary, which means that the statistics of the signal (e.g., mean and standard deviation) do not change over time. However, EEG reflects the dynamics of brain function and is inherently non-stationary. One way to solve this problem is to apply FFT to relatively stationary EEG segments. In addition, to reduce spectral leakage, a window function, such as a Hamming window or a Hanning window, can be applied to the signal before calculating the FFT.

[0049] There are also other signal processing methods to quantify frequency components and perform spectral analysis, such as wavelet transform, Hilbert transform, and matching pursuit. These methods are usually used to identify the time-frequency representation of data.

[0050] In the process of extracting features from the frequency domain representation of a signal, the distribution of power over frequency components in a given signal can be estimated based on the power spectral density. The Welch method, also known as the modified periodogram method, is a widely used method for power spectral density estimation. The Welch method (modified periodogram method) calculates the power spectral density of a signal by averaging the periodograms of overlapping smaller window segments, so it has a smaller variance compared to the periodogram of the entire epoch.

[0051] Optionally, in the process of performing Fourier transform on the electroencephalogram data to obtain the frequency-domain features of the electroencephalogram data, the features of the delta (0.5 - 4 Hz), theta (4 - 8 Hz), alpha (8 - 13 Hz), beta (13 - 40 Hz), and gamma (40 - 100 Hz) frequency bands are obtained as the frequency-domain features.

[0052] Exemplarily, since EEG has high temporal resolution, it can capture a wide range of neural oscillations. These rhythms can be divided into five standard bands: delta waves (0.5 - 4 Hz), theta waves (4 - 8 Hz), alpha waves (8 - 13 Hz), beta waves (13 - 40 Hz), and gamma waves (40 - 100 Hz). Research has shown that brain activities in each frequency band are related to different cognitive functions. These advantages make EEG not only a viable and practical option for studying important issues in neuroengineering and neuroscience but also in clinical applications and disease diagnosis processes.

[0053] Optionally, the electroencephalogram features include time-domain features; in the process of collecting electroencephalogram data of the regulation target and extracting the electroencephalogram features of the electroencephalogram data, the Hjorth method is used to quantify the signal activity degree and structure of the electroencephalogram data, calculate the activity parameter and mobility parameter of the electroencephalogram data, and obtain the time-domain features of the electroencephalogram data.

[0054] The extraction of time-domain features of EEG signals can be done without any transformation. Zero crossing is a time-domain feature that represents the number of times the signal crosses zero. This measurement method and zero-crossing interval have been used in epilepsy detection, emotion recognition, and sleep staging.

[0055] The Hjorth parameters are a method for quantifying the signal activity degree and structure. It measures and classifies signals by capturing the dynamic characteristics of time-series signals. The Hjorth parameters can consist of three main components: the activity parameter, the slow variation parameter, and the fast variation parameter. They can be used to represent the activity degree of the signal, the specific variation characteristics, and the overall variation trend of the signal in order to better represent a given signal. Exemplarily, the Hjorth parameters can be constructed to describe three time-domain feature sets of a single EEG channel. These features can be activity, mobility, and complexity. Hjorth Activity (HA) is the variance of the EEG signal (i.e., signal power), representing the width of the signal. Hjorth Mobility (HM) estimates the average frequency of the signal. Hjorth Complexity (HC) estimates the bandwidth of the EEG signal by calculating the mobility of the first derivative of the EEG relative to the EEG itself. The Hjorth parameters can be obtained by calculating the standard deviation of the signal and the standard deviations of the first and second derivatives.

[0056] Hjorth parameters can be used to detect the abnormal activity level and critical points of time series in the time series analysis of electroencephalogram data; can be used to analyze and model various signals in the signal processing process of electroencephalogram data; and can be used to discover neuropathological changes and detect the dynamic characteristics of electroencephalogram signals in the EEG signal analysis process of electroencephalogram data.

[0057] Optionally, the electroencephalogram features include entropy features; in the process of collecting and regulating the electroencephalogram data of the target and extracting the electroencephalogram features of the electroencephalogram data, the time series complexity of the electroencephalogram data is quantified based on approximate entropy and sample entropy to obtain the entropy features of the electroencephalogram data.

[0058] Electroencephalogram entropy feature refers to measuring the irregularity and information processing ability of the brain system by analyzing electroencephalogram (EEG) signals. The entropy value reflects the complexity and information processing ability of brain activities and is usually used to monitor the depth of anesthesia and evaluate the brain function state. Spectral entropy characterizes the complexity or regularity of EEG. To calculate spectral entropy, the probability distribution of the signal is approximated to its power spectral density. Then the spectral entropy can be calculated. Exemplarily, sample entropy is a commonly used method, which calculates the entropy value by analyzing the complexity of EEG signals. Sample entropy does not require coarse-graining of the original data and is suitable for the analysis of mixed signals. For example, approximate entropy (ApEn) of electroencephalogram is a method for measuring the complexity of time series and is especially suitable for analyzing electroencephalogram signals (EEG). Approximate entropy is mainly used to describe the irregularity and complexity of time series

[0059] Exemplarily, during anesthesia, the change of electroencephalogram entropy value can reflect the depth of anesthesia. For example, when the electroencephalogram entropy value is lower than 60, it usually indicates a deeper anesthesia depth; when it is lower than 40, it may enter a state of deep anesthesia or even loss of consciousness, and at this time, special attention needs to be paid to the patient's vital signs. In the waking state, the electroencephalogram entropy value usually remains between 80 and 100, and exceeding this range may indicate an abnormal brain function state, which requires further examination and evaluation.

[0060] Optionally, in the process of constructing the neural network model and obtaining the stimulation parameters according to the electroencephalogram features, it includes:

[0061] Construct the electroencephalogram features of electroencephalogram data into training vectors, and train the neural network model to establish a non-linear mapping between the electroencephalogram features and the stimulation parameters;

[0062] Based on the trained neural network model, optimize the parameters of the neural network model, and obtain the stimulation parameters according to the electroencephalogram features of the regulation target.

[0063] Figure 2Schematic diagram of the construction logic of a neural network model according to an embodiment of the present application.

[0064] Machine learning is a set of algorithms that can automatically identify patterns in data and make predictions on newly observed measurements. In the field of neuroscience research, the following situations often occur: ① comparing various conditions, ② diagnosing diseases, and ③ identifying electrophysiological changes related to behavior. As Figure 2 shown, although the specific applications are different, the process of machine learning is similar in most cases. Generally, machine learning has two stages: training and testing. In the training stage, there is a set of available examples (i.e., data with corresponding labels). Using a given machine learning algorithm, the example data is used to train the model (i.e., adjust its parameters) so that it can identify the relationship between the input data and the labels. In the testing stage, the input data without labels undergoes preprocessing, feature extraction, and feature reduction using the same methods as in the training stage, and the trained model estimated in the training stage predicts the output (i.e., the label). The main objective of the training stage is to estimate the model with the maximum prediction performance during testing. In one embodiment, the time-domain features, frequency-domain features, and entropy features of electroencephalogram data are constructed into training vectors, and a non-linear mapping between electroencephalogram features and stimulation parameters is established based on a neural network model, thereby optimizing the parameters of the neural network model. According to the electroencephalogram features of the regulation target, the stimulation parameters are obtained, so as to perform auricular nerve stimulation on the regulation target.

[0065] Optionally, the closed-loop neuromodulation method of the multimodal biofeedback further includes:

[0066] S40: Extract the electromyogram data and electrocardiogram data before and after stimulation, and perform multimodal feature calculation to quantify the regulation effect.

[0067] By continuously quantifying the regulation effect to track the auricular nerve stimulation situation, continuously adjusting the auricular nerve modulation parameters, and dynamically optimizing according to individual physiological feedback, the regulation targeting can be further improved.

[0068] Optionally, in the process of extracting the electromyogram data and electrocardiogram data before and after stimulation and performing multimodal feature calculation to quantify the regulation effect, it includes:

[0069] Obtain the standard deviation of consecutive R-R intervals according to the electrocardiogram data, and calculate the electrocardiogram time-domain index to evaluate the autonomic nerve activity of the regulation target;

[0070] Calculate the muscle activation degree according to the electromyogram data, obtain the root mean square value of the electromyogram of the target muscle group signal, quantify the improvement value of the quantitative motor function of the regulation target, and obtain the electromyogram index;

[0071] Based on the electrocardiogram time-domain index and the electromyogram index, calculate the stimulation effect score for the regulation target based on a preset model.

[0072] The root mean square value (RMS) of electromyogram is directly related to the energy of the electromyogram signal. The R-R interval refers to the time interval between two heartbeats. Exemplarily, evaluating the stimulation effect through electrocardiogram time-domain indexes and electromyogram indexes can better reflect the individual physiological conditions of the regulation target.

[0073] Optionally, in the process of calculating the stimulation effect score for the regulation target based on the electrocardiogram time-domain index and the electromyogram index according to a preset model, the calculation is performed according to the following expression:

[0074]

[0075] where ΔHRV is the change in the electrocardiogram time-domain index; HRV base is the baseline value of the electrocardiogram time-domain index; EMG post is the electromyogram index after stimulation; EMG pre is the electromyogram index before stimulation; w1 and w2 are weight values.

[0076] Second Embodiment

[0077] The present application further provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a multi-modal biofeedback closed-loop neuromodulation method as described above are implemented.

[0078] A multi-modal biofeedback closed-loop neuromodulation method and a storage medium provided by the present application collect electroencephalogram data of a regulation target, extract electroencephalogram features of the electroencephalogram data; construct a neural network model, and obtain stimulation parameters according to the electroencephalogram features; based on the stimulation parameters, stimulate the ear nerves of the regulation target through ear nerve modulation technology; based on the high-precision nerve activity reference signal provided by scalp electroencephalogram, through multi-modal data fusion, provide prior knowledge for the adaptive adjustment of the ear regulation parameters, can be dynamically optimized according to individual physiological feedback, greatly reduce the volatility of the treatment effect, achieve precise regulation and real-time closed-loop feedback, enhance the targeting of ear nerve modulation, and improve the user experience.

[0079] It should be noted that in the present application, step codes such as S10 and S20 are used. The purpose is to more clearly and briefly express the corresponding content and do not constitute a substantial limitation in sequence. Those skilled in the art may execute S20 first and then S10 during specific implementation, etc., but these should all be within the protection scope of the present application.

[0080] In the embodiments of the device and storage medium provided by the present application, all technical features of any of the above method embodiments may be included. The expansion and explanation content of the specification is basically the same as that of the above method embodiments and will not be repeated here.

[0081] An embodiment of the present application also provides a computer program product, which includes computer program code. When the computer program code runs on a computer, the computer is enabled to execute the methods in the above various possible embodiments.

[0082] An embodiment of the present application also provides a chip, which includes a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that a device equipped with the chip executes the methods in the above various possible embodiments.

[0083] It can be understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided by the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, as is known to those of ordinary skill in the art, with the evolution of device architectures and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0084] The serial numbers of the above embodiments of the present application are only for description and do not represent the superiority or inferiority of the embodiments.

[0085] The steps in the method of the embodiment of the present application can be adjusted, combined, and deleted according to actual needs.

[0086] The units in the device of the embodiment of the present application can be combined, divided, and deleted according to actual needs.

[0087] In the present application, for the description of the same or similar term concepts, technical solutions, and / or application scenarios, generally only a detailed description is given when it first appears. When it appears repeatedly later, for the sake of brevity, it is generally not described again. When understanding the technical solutions and other contents of the present application, for the same or similar term concepts, technical solutions, and / or application scenarios that are not described in detail later, reference can be made to the relevant detailed descriptions before.

[0088] In the present application, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0089] The technical features of the technical solutions of the present application can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combinations of these technical features do not conflict, they should all be considered as the scope recorded in the present application.

[0090] The above are only the preferred embodiments of the present application, and thus do not limit the scope of the present application. Any equivalent structural or equivalent process transformations made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are similarly included in the scope of protection of the present application.

Claims

1. A closed-loop neuromodulation method for multimodal biofeedback, characterized in that, Including: S10: Collect the electroencephalogram data of the regulation target, and extract the electroencephalogram features of the electroencephalogram data; S20: Construct a neural network model, and obtain the stimulation parameters according to the electroencephalogram features; S30: Based on the stimulation parameters, stimulate the ear nerves of the regulation target through the ear nerve regulation technology.

2. The closed-loop neuromodulation method for multimodal biofeedback according to claim 1, wherein The electroencephalogram features include frequency domain features; In the process of collecting the electroencephalogram data of the regulation target and extracting the electroencephalogram features of the electroencephalogram data, perform Fourier transform on the electroencephalogram data to obtain the frequency domain features of the electroencephalogram data.

3. A closed-loop neuromodulation method for multimodal biofeedback according to claim 2, characterized in that, In the process of performing Fourier transform on the electroencephalogram data to obtain the frequency domain features of the electroencephalogram data, obtain the features of the δ wave (0.5 - 4 Hz), θ wave (4 - 8 Hz), α wave (8 - 13 Hz), β wave (13 - 40 Hz), and γ wave (40 - 100 Hz) frequency bands as the frequency domain features.

4. A closed-loop neuromodulation method for multimodal biofeedback according to claim 1, characterized in that, The electroencephalogram features include time domain features; In the process of collecting the electroencephalogram data of the regulation target and extracting the electroencephalogram features of the electroencephalogram data, use the Hjorth method to quantify the signal activity degree and structure of the electroencephalogram data, calculate the activity parameter and mobility parameter of the electroencephalogram data, and obtain the time domain features of the electroencephalogram data.

5. A closed-loop neuromodulation method for multimodal biofeedback according to claim 1, characterized in that, The electroencephalogram features include entropy features; In the process of collecting the electroencephalogram data of the regulation target and extracting the electroencephalogram features of the electroencephalogram data, quantify the time series complexity of the electroencephalogram data based on approximate entropy and sample entropy to obtain the entropy features of the electroencephalogram data.

6. A closed-loop neuromodulation method for multimodal biofeedback according to claim 1, characterized in that, In the process of constructing the neural network model and obtaining the stimulation parameters according to the electroencephalogram features, it includes: Construct the electroencephalogram features of the electroencephalogram data into a training vector, and train the neural network model to establish a non-linear mapping between the electroencephalogram features and the stimulation parameters; Based on the trained neural network model, optimize the parameters of the neural network model, and obtain the stimulation parameters according to the electroencephalogram features of the regulation target.

7. A closed-loop neuromodulation method for multimodal biofeedback according to any one of claims 1-6, characterized in that, The closed-loop nerve regulation method of multi-modal biofeedback further includes: S40: Extract the electromyogram data and electrocardiogram data before and after stimulation, and perform multi-modal feature calculation to quantify the regulation effect.

8. A closed-loop neural regulation method for multimodal biofeedback according to claim 7, characterized in that In the process of extracting the electromyogram data and electrocardiogram data before and after stimulation and performing multi-modal feature calculation to quantify the regulation effect, it includes: Obtain the standard deviation of consecutive R-R intervals according to the electrocardiogram data, calculate the electrocardiogram time domain index to evaluate the autonomic nerve activity of the regulation target; Calculate the muscle activation degree according to the electromyogram data, obtain the root mean square value of the electromyogram of the target muscle group signal, quantify the improvement value of the quantitative motor function of the regulation target, and obtain the electromyogram index; Based on the electrocardiogram time domain index and the electromyogram index, calculate the stimulation effect score for the regulation target based on a preset model.

9. A closed-loop neuromodulation method for multimodal biofeedback according to claim 8, characterized in that, In the process of calculating the stimulation effect score for the regulation target based on the electrocardiogram time domain index and the electromyogram index according to the following expression: Among them, ΔHRV is the change amount of the electrocardiogram time-domain index; HRV base is the baseline value of the electrocardiogram time-domain index; EMG post is the electromyogram index after stimulation; EMG pre is the electromyogram index before stimulation; w1 and w2 are weight values.

10. A storage medium, characterized in that, A computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of a closed-loop neuromodulation method for multimodal biofeedback as described in any one of claims 1-9 are implemented.

Images