Self-adaptive closed-loop electroencephalogram regulation and control device and method based on multi-modal feedback

Abstract

According to the self-adaptive closed-loop electroencephalogram regulation and control device and method based on multi-modal feedback, multi-modal sensing and fusion are adopted, and a system does not depend on single electroencephalogram signals and fuses data of three modals of electroencephalogram, temperature and aromatherapy. The fusion provides a more comprehensive and more stereoscopic user state portrait, overcomes the defects that a single signal is easy to interfere and information is one-sided, and greatly improves the accuracy of state recognition (such as relaxation, anxiety and sleep stages).

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electroencephalogram regulation, and in particular to an adaptive closed-loop electroencephalogram regulation device and method based on multi-modal feedback. BACKGROUND

[0002] In the field of non-invasive neural regulation, existing researches widely use wearable electroencephalogram acquisition devices to obtain electroencephalogram (EEG) through scalp electrodes for applications such as emotion recognition, attention assessment, and sleep state monitoring. Existing portable electroencephalogram devices mainly use physical stimuli such as electricity, magnetism, light, and sound, which are difficult to meet complex physiological needs and lack the ability to actively regulate the microenvironment of the head (temperature / odor). SUMMARY

[0003] Therefore, the present application aims to provide an adaptive closed-loop electroencephalogram regulation method based on multi-modal feedback to solve some or all of the problems in the background art.

[0004] To achieve the above purpose, the present application provides an adaptive closed-loop electroencephalogram regulation method based on multi-modal feedback, comprising the following steps:

[0005] a comb-shaped dry electrode array arranged on the forehead and temporal lobe region of the head for collecting multi-channel electroencephalogram signals;

[0006] a semiconductor refrigeration plate co-located with the comb-shaped dry electrode for adjusting the temperature of the head;

[0007] a temperature sensor for real-time monitoring of scalp temperature;

[0008] an aromatherapy micro-pump device for storing and controllably releasing aromatherapy molecules;

[0009] a signal acquisition and preprocessing module connected to the head-mounted device for filtering and denoising the collected electroencephalogram signals, and standardizing the temperature signals;

[0010] a multi-modal feature extraction and state recognition module for fusion analysis of the preprocessed electroencephalogram signals, temperature data, and aromatherapy concentration data based on a time-domain differential convolution algorithm and a nonlinear heterogeneous graph fusion model, and outputting user physiological state recognition results;

[0011] a reinforcement learning closed-loop control module for generating aromatherapy release control instructions and temperature adjustment instructions based on a preset reward function through a policy network according to the physiological state recognition results, and sending them to the head-mounted device for execution;

[0012] a user data management module for storing user historical data, personalized schemes, and scene mode libraries.

[0013] Optionally, at least one electrode in the comb-shaped dry electrode array is a multiplexed electrode, which has both the functions of collecting electroencephalogram signals and conducting temperature regulation.

[0014] Optionally, the nonlinear heterogeneous graph fusion model defines the electroencephalogram features, scalp temperature, and aromatherapy concentration as different node types in the heterogeneous graph, and defines the edges based on the physical and physiological relationship between the nodes, and performs feature fusion through a graph neural network.

[0015] Optionally, the time-domain differential convolution algorithm includes first-order differential processing of the input electroencephalogram signals and temperature signals, and convolution operation using a convolution kernel matched to the characteristics of each signal to extract dynamic features.

[0016] Optionally, the reinforcement learning closed-loop control module adopts a deep deterministic policy gradient (DDPG) algorithm, the state space of which includes electroencephalogram frequency band power, temperature gradient, and aromatherapy concentration, the action space of which includes aromatherapy pulse change and temperature change instruction, and the reward function of which integrates deep sleep promotion index, temperature steady-state index, and aromatherapy accuracy index.

[0017] Optionally, the multi-modal feedback-based reinforcement learning closed-loop regulation system further comprises a scene adaptation module for automatically calling a corresponding stimulation mode library according to user identity or environmental information, the stimulation mode library including a first mode for beauty relaxation, a second mode for traditional Chinese medicine physiotherapy, and a third mode for psychological anxiety intervention.

[0018] Based on the same inventive concept, the present application also provides a multi-modal feedback-based reinforcement learning closed-loop regulation method, applied to the multi-modal feedback-based reinforcement learning closed-loop regulation system as described above, the method comprising:

[0019] Synchronously collecting electroencephalogram signals and head temperature data of a user through an integrated head-mounted device;

[0020] Pretreating the collected electroencephalogram and temperature signals;

[0021] Extracting dynamic features of the signals using a time-domain differential convolution algorithm;

[0022] Inputting the extracted features and real-time aromatherapy concentration data into a nonlinear heterogeneous graph fusion model for multi-modal feature fusion and user physiological state recognition;

[0023] Inputting the recognized physiological state into a pre-trained reinforcement learning control model to generate aromatherapy release control instructions and / or temperature regulation instructions;

[0024] Sending the instructions to the head-mounted device for execution to complete a regulation cycle;

[0025] Based on the newly collected physiological data after execution, the reward value is calculated and the reinforcement learning control model strategy is updated to realize closed-loop adaptive optimization.

[0026] Optionally, the training process of the reinforcement learning control model comprises: generating continuous control actions through a policy network and evaluating and optimizing the actions through a value network with the goal of maximizing cumulative rewards, wherein the reward function is determined by the deep sleep electroencephalogram power change, the deviation of the scalp temperature from the target temperature, and the deviation of the incense concentration from the target concentration.

[0027] Optionally, before generating the control instructions, the method further comprises: calling a stimulation mode library matched with the current user scenario to set initial parameters or constraint ranges of the action space of the reinforcement learning model.

[0028] Based on the same inventive concept, the application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above methods.

[0029] As can be seen from the above, the adaptive closed-loop electroencephalogram regulation system based on multi-modal feedback provided by the application adopts multi-modal perception and fusion. Instead of relying on a single electroencephalogram signal, the system fuses data of three modalities of electroencephalogram (physiological state), temperature (physical state), and incense (chemical stimulation). This fusion provides a more comprehensive and three-dimensional user state portrait, overcoming the shortcomings of single signal interference and information fragmentation, and greatly improving the accuracy of state recognition (such as relaxation, anxiety, and sleep stage). The nonlinear heterogeneous graph fusion model is adopted. This advanced algorithm can intelligently capture complex and nonlinear physiological-physical relationships between different modal data (such as the promotion of a certain incense to alpha brain waves at a specific temperature). This reveals the deep state correlation better than simple linear weighted fusion, providing a solid decision basis for precise regulation. Reinforcement learning closed-loop adaptive: the system is not a static, one-size-fits-all stimulator. It can adjust the strategy in real time according to the actual effect (reward value) after each intervention. This means that the system can continuously learn and adapt to the unique physiological response patterns of each user (for example, user A is sensitive to lavender, while user B may need different temperature settings), truly realizing the personalized adaptive optimization of "the more you use, the smarter you get", and achieving better long-term results. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the present application or related art, the following will briefly introduce the drawings needed to be used in the embodiments or related art descriptions. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0031] Figure 1 A schematic diagram of an adaptive closed-loop electroencephalogram regulation system framework based on multi-modal feedback according to an embodiment of the present application;

[0032] Figure 2 A schematic diagram according to an embodiment of the present application;

[0033] Figure 3 A state recognition model training process flow diagram according to an embodiment of the present application;

[0034] Figure 4 A schematic diagram of an adaptive closed-loop electroencephalogram regulation method flow according to an embodiment of the present application;

[0035] Figure 5 A schematic diagram of electrode layouts according to an embodiment of the present application;

[0036] Figure 6 A schematic diagram of an electronic device hardware structure according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions, and advantages of the present application clearer, further detailed explanations will be given below with reference to the embodiments and the accompanying drawings.

[0038] It should be noted that, unless otherwise defined, technical terms or scientific terms used in the embodiments of the present application should be understood as their common meanings to those of ordinary skill in the art to which the present application pertains. The terms “first”, “second”, and similar terms used in the embodiments of the present application do not represent any order, number, or importance, but are only used to distinguish different components. The terms “include”, “contain”, and similar terms mean that the components or objects before the terms encompass the components or objects listed after the terms and their equivalents, without excluding other components or objects. The terms “connect” or “connected” and similar terms do not mean only physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper”, “lower”, “left”, “right”, and the like only represent relative positional relationships, which can change when the absolute positions of the described objects change.

[0039] Based on the background art, a common electroencephalogram acquisition system usually contains 8-16 channels, covering key areas such as frontal lobe, temporal lobe, and parietal lobe, and uses Ag / AgCl electrodes or dry electrodes to achieve signal acquisition. At the same time, existing portable electroencephalogram devices mainly use physical stimuli such as electricity, magnetism, light, and sound, which are difficult to meet complex physiological needs and lack the ability to actively regulate the microenvironment (temperature / odor) of the head.

[0040] In terms of data processing, traditional electroencephalogram analysis methods rely on frequency domain feature extraction (such as alpha wave and beta wave power calculation) and time domain statistical analysis (such as mean and variance). In recent years, with the development of deep learning, convolutional neural networks (CNN), recurrent neural networks (RNN), and Transformer architecture have been widely used in electroencephalogram signal classification and emotion recognition tasks. For example, LSTM models can effectively capture the time series features of electroencephalogram signals, while Transformers are good at modeling spatial correlations across channels.

[0041] However, most existing systems still use an "open-loop" control mode, i.e., only setting the stimulation parameters based on the initial state, lacking real-time feedback mechanisms to dynamically adjust the stimulation strategy. This static control method cannot meet the individualized treatment needs and reduces the stability and consistency of clinical effects.

[0042] To solve the above technical problems, as Figure 1 The application provides a reinforcement learning closed-loop control system based on multi-modal feedback, which comprises a comb-shaped dry electrode array 101, a semiconductor cooling fin 102, a temperature sensor 103, a fragrance micro-pump device 104, a signal acquisition and preprocessing module 105, a multi-modal feature extraction and state recognition module 106, a reinforcement learning closed-loop control module 107, and a user data management module 1018.

[0043] The electroencephalogram acquisition instrument adopts the structure as shown in Figure 2 The structure comprises left frontal lobe comb electrodes, right frontal lobe comb electrodes, left temple temperature control electrodes, right temple temperature control electrodes, left electroencephalogram dry electrodes, right electroencephalogram dry electrodes, left fragrance pools, right fragrance pools, and a breathing expander. Figure 2 In the middle 4, 1, 5, the frontal electrodes use flat dry electrodes (Fp1, Fpz, Fp2), and 1->Fpz is a multiplexed electrode that can sample frontal electroencephalogram data in time-sharing mode. Meanwhile, it is connected with a heating semiconductor inside, which can be used to adjust the temperature of the forehead according to the algorithm, and a temperature sensor is built-in. Figure 2 In the middle 2, 3, the temple is considered to have hair, so short protruding comb dry electrodes are used, which are multiplexed electrodes that can be used to sample frontal electroencephalogram waves and adjust head temperature. The surface has concave and convex shapes to reduce the impact of short hair at the temple on contact. The front side is connected to left incense 7, 8 for linkage control of the emission of incense. Figure 2 In the middle 1, 2, the frontal lobe and the top of the head where the hair is long are used with comb normal comb dry electrodes, and the distance can be adjusted.

[0044] The upper computer analysis software uses C# to make the analysis software portable, which can call a Python-based electroencephalogram big model analysis inside, has user management, customer management, real-time state display, and historical data analysis and fact data analysis functions.

[0045] The specific electrode layout is shown in the following table Figure 5 .

[0046] Comb-shaped dry electrode array 101: Arranged on the user's forehead and bilateral temporal lobe regions, with a non-invasive design, adjustable electrode spacing to accommodate different head shapes; electrode material is conductive silicone or silver / silver chloride composite material to ensure signal quality and comfort; the number of electrodes is 8 to 32, supporting multi-channel synchronous acquisition, with a sampling rate not less than 250Hz.

[0047] Semiconductor cooling sheet 102: Integrated into the electrode array substrate, local heating or cooling is achieved through the Peltier effect, with a temperature regulation range of 18°C to 42°C, an accuracy of ±0.5°C, and a response time of less than 10 seconds.

[0048] Temperature sensor 103: Using NTC thermistor or infrared sensor, arranged near the electrodes, with a sampling frequency of 1Hz, real-time feedback of scalp temperature changes, data transmission through I2C or SPI interface.

[0049] Aromatherapy micro-pump device 104: Contains a micro-piezoelectric pump or a micro-fluid control unit, supporting independent storage and on-demand release of various essential oils (such as lavender, rosemary, etc.), with a release accuracy of ±0.1μL / time, supporting pulse or continuous release mode.

[0050] Temperature regulation directly affects the transdermal absorption efficiency of aromatherapy molecules (35% absorption rate increase at elevated temperature) and neural excitability (40% theta wave power decrease at reduced temperature), thereby affecting the output of brain waves, and treating and relieving fatigue, depression, and anxiety. Regulation process: reduce temperature to inhibit neural excitability when fatigued or anxious; increase temperature to promote absorption when relaxed.

[0051] Signal acquisition and preprocessing module 105: Integrates analog front end (AFE) such as ADS1299, supports high common mode rejection ratio (CMRR > 110dB) and programmable gain; filtering includes 0.5-45Hz band-pass filtering, 50 / 60Hz notch filtering; denoising uses ICA or wavelet transform; temperature signal is normalized and moving average processed.

[0052] Multi-modal feature extraction and state recognition module 106: Runs on an embedded processor (such as ARM Cortex-A series) or a cloud server, extracts features such as δ, θ, α, β band power, asymmetry, and coherence of EEG; temperature features include gradient change rate and local temperature difference; essential oil concentration is in ppm; heterogeneous graph fusion is performed through graph neural network (GNN) or Transformer structure, outputting states such as "relaxed", "anxious", "light sleep", "deep sleep", etc.

[0053] Reinforcement learning closed loop control module 107: using DDPG, SAC or PPO algorithm, the state space includes EEG feature vector, temperature sequence, and historical release amount of incense; the action space is temperature setting value change and incense pump switch frequency; the reward function comprehensively considers sleep quality index, temperature comfort, and incense utility satisfaction.

[0054] Construction of closed loop control module: the first step is physiological data acquisition and preprocessing

[0055] The collected objects include users wearing wearable electroencephalogram devices, the device has 24-channel whole brain electroencephalogram signal acquisition capability, and simultaneously acquires electrocardiogram and respiration signals to form a multi-modal physiological data set. In order to facilitate subsequent analysis and modeling, the first step is to standardize and preprocess the original data; the controller's preprocessing model is used to remove head motion artifacts from eye movement data, and temperature data is compensated according to environmental temperature; the baseline drift of respiration data is corrected; power frequency filtering is performed on electrocardiogram data; ICA independent component analysis is used to remove eye movement artifacts from electroencephalogram data. Electroencephalogram signal preprocessing: band-pass filtering range: 0.5Hz~30Hz; wavelet denoising or ICA independent component analysis to remove eye movement artifacts; extract frequency band energy (such as alpha wave 8~12Hz, beta wave 13~30Hz). Electrocardiogram signal preprocessing: 50Hz power frequency interference elimination; R wave detection and heart rate variability (HRV) extraction. Respiration signal preprocessing: filter smoothing processing; extract respiration frequency and depth features. Finally, the data is formed into a structured data table, the fields include the following table:

[0056] ID User ID Acquisition timestamp EEG channel voltage values Heart rate Respiration rate Emotion score (SAS / SDS scale) Sleep quality score (PSQI scale) Remarks 1 XX-001 2

[0057] The second step is multi-modal data analysis and state recognition

[0058] The collected multi-modal physiological data is analyzed by a deep learning algorithm based on time domain differential convolution feature extraction + nonlinear graph fusion (multi-modal heterogeneous graph fusion of electroencephalogram features, incense concentration, and scalp temperature) to recognize the current user's neural state (such as anxiety, depression, fatigue, and insomnia), which serves as the basis for subsequent stimulus strategy development. The specific implementation process is divided into three stages:

[0059] (1) The first stage: data annotation and training sample library construction

[0060] Sample collection: collect electroencephalogram, electrocardiogram, and respiration data of patients with depression, anxiety, and insomnia from clinical trials.

[0061] Sample standardization: define the electroencephalogram frequency band change pattern corresponding to different diseases (such as decreased alpha wave and increased theta wave in patients with insomnia).

[0062] Annotation rules: each sample is annotated with disease type, severity, and treatment response.

[0063] Sample Augmentation: Use time series augmentation techniques (e.g., adding white noise, temporal perturbation, channel random dropout) to expand the dataset.

[0064] (2) Second stage: State recognition model construction and training

[0065] Build a multi-modal fusion model, input is the time series data of EEG, ECG, and respiration.

[0066] Use LSTM+Transformer architecture to extract deep temporal dependencies and cross-modal features.

[0067] The output layer is a three-class or multi-class result (e.g., normal, mild anxiety, moderate depression, insomnia, etc.).

[0068] Introduce transfer learning in the training process, use public psychological disease datasets (e.g., DEAP, SEED) as pre-training data source to improve generalization ability.

[0069] Where, the convolution kernel function Where, g: is the function name defined. Here, it specifically refers to "Gaussian first derivative function". t is the independent variable of the function. d / dt is the derivative operator. It means to take the first derivative of the function e^{-t^2 / 2\sigma^2} with respect to the variable t. Therefore, the entire function g(t) is the derivative of the exponential function behind. e is a natural constant (approximately equal to 2.71828). σ (sigma): This is the Greek letter, representing the standard deviation. It is a very important parameter in this function. σ controls the width of the Gaussian function, and thus controls the shape of its derivative. The larger the σ value, the wider and flatter the function curve. The smaller the σ value, the narrower and sharper the function curve.

[0070] Graph fusion formula Where, is the final output of the formula. It represents the normalized attention weight, i.e. the importance of node j to node i. The sum of all neighbor node j α_ij is equal to 1.

[0071] Softmax() is a normalized exponential function. Its role is to normalize the attention coefficients (e_ik, k ∈ N(i)) flowing into node i, converting it into a probability distribution (the sum of all weights is 1), so that the attention weights between different nodes can be compared.

[0072] LeakyReLU() activation function. Its role here is to introduce a weak nonlinearity for the attention mechanism. Its characteristic is that for negative input values, there will be a small slope (such as 0.2), rather than being directly set to zero like standard ReLU, which helps the model learn more rich information. a is a learnable parameter vector (usually a one-dimensional tensor). It is the core parameter of the graph attention mechanism, responsible for mapping the concatenated high-order features to a scalar value (i.e., attention coefficient). T transpose symbol. Indicates that the vector a is transposed to perform matrix multiplication (dot product) with the following vector. W is a learnable weight matrix. It acts on the original feature vector of each node and is a shared parameter that performs linear transformation on all nodes to map the input features to a higher or lower dimensional space (feature transformation / encoding). i and j Original feature vector of node i and node j (or output feature of the previous layer). This is the initial input of the graph data.

[0073] (3) Third stage: ROI region identification and local feature extraction

[0074] • Extract the activity features of the local brain region for key regions in the EEG data (such as the prefrontal cortex).

[0075] Use the attention mechanism to focus on the most discriminative channels and time periods to assist in subsequent stimulus target selection.

[0076] Step 3: Closed-loop regulation and parameter optimization

[0077] After completing user state recognition, the system enters the closed-loop regulation stage, automatically calls the corresponding stimulus mode library, and adjusts the stimulation parameters through real-time feedback to achieve dynamic intervention.

[0078] 1. LSTM sleep stage prediction model

[0079] Input:

[0080] Time domain differential features: $\frac{dEEG}{dt}$, $\frac{dTEMP}{dt}$, respiratory disturbance index

[0081] Contextual features: current sleep stage label (Wake / N1 / N2 / N3 / REM)

[0082] Python structure design:

[0083] model = Sequential()

[0084] model.add(LSTM(128, input_shape=(300, 15), return_sequences=True)) # 5-minute timing window (10Hz×300s)

[0085] model.add(LSTM(64))

[0086] model.add(Dense(32, activation='relu'))

[0087] model.add(Dense(1, activation='sigmoid')) # Output the probability of deep sleep

[0088] Pre-intervention logic: When $P(N3)_{t+5min}>0.85$, the aromatherapy micro-pump is triggered to release delta waves to promote the formulation (lavender + sandalwood, concentration gradient increase).

[0089] 2. Temporal Differential Convolution Feature Extraction

[0090] Operating procedures:

[0091] Taking the first-order differential of the original signal $X(t)$: $\nabla X = X(t) - X(t-1)$

[0092] Convolution kernel design:

[0093] EEG: Gabor nucleus (preferably 1-4Hz delta wave)

[0094] Temperature: Gaussian difference kernel (for extracting fast fluctuations)

[0095] Breathing: Rectangular nucleus (captures apnea events)

[0096] Output feature map: $F_{conv} = ReLU(\nabla X \ast KERNEL)$

[0097] 3. Nonlinear heterogeneous graph fusion (core technology)

[0098] Graph structure definition:

[0099] Node type: ${EEG, TEMP, SCENT, EYE}$ (eye tracking)

[0100] Edge Relationship:

[0101] $EEG \xrightarrow{\partial t / \partial T} TEMP$ (Temperature gradient affects EEG)

[0102] $SCENT \xrightarrow{HRV} ECG$ (Aromatherapy concentration regulates the autonomic nervous system)

[0103] Fusion equation:

[0104] H(l+1) = σ(∑r∈R ∑ArH(l)Wr(l)) where Ar = adjacency matrix, Wr = relation-specific weight.

[0105] 4. Strengthen learning closed-loop control

[0106] State space:

[0107] $S_t = [\alpha_{power}, \beta / \alpha, \Delta T_{max}, scent_{ppm},blink_{rate}]$

[0108] Action space:

[0109] Aroma pulse: $\Delta scent \in {-0.1, 0, +0.1}\ ppm / s$

[0110] Temperature control command: $Peltier \in {-0.5, 0, +0.5}\ ^\circ C$

[0111] Reward function:

[0112] ·Rt=w1Δδpower Deep sleep promotion −w2∣Tscalp−36.5∣ Temperature steady state +w3e−∣scent−opt∣ Aromatherapy Precision Rt = Deep Sleep Promotion w1Δδpower−Temperature Steady-State w2∣Tscalp−36.5∣+Aromatherapy Precision w3e−∣scent−opt∣

[0113] Policy Network: Continuous control is optimized using DDPG (Deep Deterministic Policy Gradient).

[0114] The fourth step is to adapt to multiple scenarios and generate personalized treatment plans.

[0115] Based on the needs of different application scenarios (such as beauty salons, traditional Chinese medicine physiotherapy centers, and psychological counseling institutions), the system has multiple built-in stimulation mode libraries and supports personalized customization.

[0116] The above stimulation modes can include: 1. Transcranial Electrical Stimulation (TES)

[0117] 1. Transcranial Direct Current Stimulation (tDCS)

[0118] Principle: Adjust the excitability of cortical neurons through constant low-intensity direct current (0-2mA). Anodal stimulation enhances neuronal activity, and cathodal stimulation inhibits activity.

[0119] Mode: Includes pure direct current mode, sine wave mode, square wave pulse mode, pseudo-noise mode, etc.

[0120] Application: Improve cognitive function (such as attention, working memory), treat depression, Parkinson's disease, post-stroke movement disorders, etc.

[0121] Advantages: Non-invasive, easy to operate, low cost, compatible with EEG, fMRI, etc. Multimodal research.

[0122] 2. Transcranial Alternating Current Stimulation (tACS)

[0123] Principle: Use frequency-adjustable alternating current to synchronize or interfere with brain neural oscillations to regulate specific brain functions.

[0124] Application: Enhance cognitive flexibility, improve sleep quality, treat neuropathic pain, etc.

[0125] Features: Different frequencies (such as Alpha waves, Gamma waves) can specifically affect different brain region activities.

[0126] 3. Transcranial Random Noise Stimulation (tRNS)

[0127] Principle: Apply random noise form of current to increase neuronal excitability threshold and improve neural plasticity.

[0128] Application: Promote motor learning, improve language function, assist brain injury rehabilitation.

[0129] Advantages: Stimulation effect is long-lasting, suitable for long-term neural function reconstruction.

[0130] 4. Transcranial Pulse Electric Stimulation (tPCS)

[0131] Principle: Stimulate the brain through short pulse current to adjust neuronal membrane potential.

[0132] Application: Relieve chronic pain, improve mood disorders.

[0133] Features: Pulse parameters (such as width, frequency) can be flexibly adjusted to adapt to different treatment needs.

[0134] II. Deep Brain Stimulation (DBS)

[0135] 1. Principle: Implantation of electrodes into specific nuclei in the brain (e.g., subthalamic nucleus, internal segment of globus pallidus) through stereotactic surgery to release high-frequency electrical pulses to suppress abnormal neural activity.

[0136] 2. Modes:

[0137] High-frequency stimulation (usually >100 Hz): Inhibits overactive neurons, improving motor symptoms.

[0138] Low-frequency stimulation (<50 Hz): May enhance neuronal activity, used for specific pain or epilepsy treatment.

[0139] 3. Applications:

[0140] Movement disorders: Parkinson's disease tremor, rigidity, and bradykinesia, dystonia.

[0141] Mental disorders: Obsessive-compulsive disorder, depression (FDA-approved).

[0142] Pain management: Intractable nociceptive pain, neuropathic pain.

[0143] 4. Features: Requires precise targeting of the target, postoperative individualized programming of parameters.

[0144] III. Vagus Nerve Stimulation (VNS)

[0145] 1. Principle: Stimulation of the vagus nerve in the neck through an implanted device to affect neural activity in areas such as the brain's limbic system and reticular formation.

[0146] 2. Modes:

[0147] Intermittent stimulation: Periodic release of electrical pulses (e.g., 30 seconds of stimulation / 5 minutes of rest).

[0148] Closed-loop stimulation: Combines physiological signals such as heart rate variability to dynamically adjust stimulation parameters.

[0149] 3. Applications:

[0150] Epilepsy: Drug-resistant epilepsy, reducing seizure frequency.

[0151] Depression: Adjunctive treatment of refractory depression.

[0152] Others: Exploratory treatment of migraine, inflammatory diseases (e.g., rheumatoid arthritis).

[0153] 4. Advantages: Non-direct stimulation of the brain, relatively fewer side effects.

[0154] Four, Spinal Cord Stimulation (SCS)

[0155] 1. Principle: Implant electrodes in the epidural space of the spinal cord, stimulate the dorsal column of the spinal cord through electrical pulses, block the transmission of pain signals to the brain.

[0156] 2. Mode:

[0157] Traditional stimulation: High frequency (50-100Hz) continuous stimulation, produces tingling sensation to mask pain.

[0158] High-frequency stimulation (>1kHz): Reduce tingling sensation, improve patient comfort.

[0159] Closed-loop stimulation: Combine with electromyographic signals or motor intention, dynamically adjust stimulation intensity.

[0160] 3. Application:

[0161] Chronic pain: Failed back surgery syndrome (FBSS), complex regional pain syndrome (CRPS).

[0162] Ischemic diseases: Intractable angina, peripheral arterial disease.

[0163] 4. Features: Requires surgical implantation of equipment, but can significantly reduce opioid use.

[0164] Five, Functional Electrical Stimulation (FES)

[0165] 1. Principle: Use low-frequency electrical pulses to stimulate nerves and muscles, activate muscles during functional tasks (such as grasping, walking).

[0166] 2. Mode:

[0167] Myoelectric triggered FES: Trigger electrical stimulation through surface electromyographic signals, achieve closed-loop control.

[0168] Timing stimulation: Release pulses at preset time intervals to assist in completing repetitive movements.

[0169] 3. Application:

[0170] Motor rehabilitation: Foot drop after stroke, hand function reconstruction for patients with spinal cord injury.

[0171] Prosthetic control: Drive prosthetic movement by stimulating residual limb muscles through electrical stimulation.

[0172] 4. Advantages: Directly assist in function implementation, promote neural remodeling.

[0173] Six, electroencephalographic biomimetic electrical stimulation (biomimetic bioelectric stimulation)

[0174] 1. Principle: Simulate natural bioelectric signals of the human body, through electrode patches pasted on the auricular mastoid, non-invasively introduce into the cerebellar fastigial nucleus (FN), activate the inherent neuroprotective mechanism of the brain.

[0175] 2. Mode:

[0176] Biomimetic bioelectric mode: Generate waveforms similar to healthy brain waves to replace pathological bioelectricity.

[0177] Multi-band composite mode: Combine different frequency currents to simultaneously improve blood circulation and neural function in the brain.

[0178] 3. Application:

[0179] Neurological rehabilitation: stroke, cerebral infarction, cerebral hemorrhage recovery period, brain trauma, etc.

[0180] Pain management: Vertebrobasilar insufficiency caused by migraine and cervical spondylosis.

[0181] Mental illness: Depression, anxiety.

[0182] Cognitive improvement: Senile dementia, cognitive dysfunction.

[0183] 4. Features: Non-invasive, multi-target regulation, suitable for various brain-related diseases.

[0184] User data management module 108: Supports local or cloud storage, records user's physiological data, control instructions, and status tags for each use; Supports personalized mode settings such as "pre-sleep mode" and "work focus mode"; Supports multi-user account and data isolation.

[0185] In some embodiments, at least one electrode in the comb-shaped dry electrode array is a multiplexed electrode that has both EEG signal acquisition and temperature regulation functions.

[0186] The multiplexed electrode adopts a double-layer structure: the surface layer is a conductive gel or metal coating for EEG acquisition, and the bottom layer is in direct contact with the semiconductor cooling sheet, which conducts heat through the electrode itself to achieve local temperature regulation, avoiding the placement of additional sensors and improving integration and comfort.

[0187] In some embodiments, the nonlinear heterogeneous graph fusion model defines electroencephalographic features, scalp temperature, and essential oil concentration as different node types in the heterogeneous graph, and defines edges based on the physical and physiological relationships between nodes, and performs feature fusion through a graph neural network.

[0188] The node types in the figure include: EEG channel nodes, temperature sensor nodes, and incense release nodes; the edge relationships include: spatial proximity, physiological correlation (such as the association of alpha waves with a relaxed state), and time synchronization; the GNN structure adopts GraphSAGE or GAT, and supports dynamic graph updating and attention mechanisms.

[0189] In some embodiments, the time-domain differential convolution algorithm includes: performing first-order differential processing on the input electroencephalogram signals and temperature signals, and performing convolution operations using convolution kernels matched to the characteristics of the signals to extract dynamic features.

[0190] The first-order difference is used for the EEG signal to extract transient changes, and the Gabor or Morlet wavelet kernel is used as the convolution kernel to extract time-frequency features; the first-order differential is used for the temperature signal to extract the change trend, and the Difference of Gaussian (DoG) kernel is used as the convolution kernel to highlight the temperature change rate.

[0191] In some embodiments, the reinforcement learning closed-loop control module adopts a Deep Deterministic Policy Gradient (DDPG) algorithm, the state space includes electroencephalogram frequency band power, temperature gradient, and incense concentration, the action space includes incense pulse change amount and temperature change instruction, and the reward function integrates deep sleep promotion indicators, temperature steady-state indicators, and incense precision indicators.

[0192] The state space specifically includes: delta / theta / alpha / beta power ratio, absolute value of temperature gradient, and cumulative release amount of incense; the action space is: release duration of the incense pump each time (0.1-2 seconds) and adjustment amplitude of the temperature set value (±0.1°C-1°C); and the reward function is a weighted sum: R = w1·Δδ_power + w2·(1 - |T_current - T_target|) + w3·(1 - |C_current - C_target|).

[0193] In some embodiments, a scene adaptation module is further included, which is configured to automatically call corresponding stimulation mode libraries according to user identity or environmental information, the stimulation mode libraries including a first mode for beauty relaxation, a second mode for traditional Chinese medicine physiotherapy, and a third mode for psychological anxiety intervention.

[0194] The scene adaptation module supports NFC / RFID to identify user identity, or obtains environmental information (such as time and location) through Bluetooth / WIFI; the mode library is preconfigured with multiple stimulation strategies, such as the first mode focusing on low-frequency incense and mild cooling, the second mode combining acupoint temperature stimulation, and the third mode using high-frequency incense pulse and rapid cooling to cope with acute anxiety.

[0195] Based on the same inventive concept, such as Figure 4As shown, the application also provides a reinforcement learning closed-loop regulation method based on electroencephalogram-temperature-fragrance multi-modal feedback, applied to the system as described above, the method comprising the following steps:

[0196] Step 201, synchronously collect EEG and temperature signals through the head-mounted device, sampling rate ≥ 250 Hz, temperature sampling rate ≥ 1 Hz;

[0197] Step 202, preprocessing includes: EEG filtering, de-noising, segmentation; temperature data sliding average filtering;

[0198] Step 203, time-domain differential convolution processing: first-order differential + convolution on EEG, extracting dynamic features;

[0199] Step 204, heterogeneous graph fusion: construct graph structure, input GNN model, output state probability distribution;

[0200] Step 205, reinforcement learning control: input state vector into DDPG policy network, output control instruction;

[0201] Step 206, instruction execution: control the micro-pump to release fragrance, adjust the temperature of the cooling fin;

[0202] Step 207, closed-loop update: calculate the reward according to the new round of collected data, update the policy network parameters.

[0203] In some embodiments, the training process of the reinforcement learning control model includes: generating continuous control actions through the policy network and evaluating and optimizing the actions through the value network with the goal of maximizing cumulative rewards, wherein the reward function is determined by the changes in deep sleep electroencephalogram power, the deviation of scalp temperature from the target temperature, and the deviation of fragrance concentration from the target concentration.

[0204] The training uses an experience replay mechanism, with a batch size of 128, a learning rate of 1e-4 for the policy network and 1e-3 for the value network; the weights of each term in the reward function are adjustable, supporting online learning combined with offline pre-training.

[0205] In some embodiments, before generating the control instruction, it further includes: calling a stimulation mode library matched with the current user scenario to set the initial parameters or constraint range of the action space of the reinforcement learning model.

[0206] For example, in "sleep mode", the temperature adjustment amplitude is limited to no more than ±2°C, and the fragrance release frequency is no more than 1 time / minute, to avoid excessive stimulation.

[0207] In summary, the purpose of the present application is to provide a reinforcement learning closed-loop regulation system and method based on electroencephalogram-temperature-aromatic multi-modal feedback, to make up for the closed-loop strategy of existing portable electroencephalogram regulation devices in gas stimulation, head temperature regulation, and brain wave regulation, while the patent optimizes the portable head-mounted structure, overcomes the mutual interference of stimulation channels on the collected signals, uses multi-modal real-time data to adaptively adjust the stimulation parameters, and achieves the control of the closed loop.

[0208] In particular, the present application provides a hardware integration of aromatherapy-temperature control + time-domain differential convolution algorithm + reinforcement learning closed-loop design, taking head temperature control as the core, supplemented by micro-current stimulation + multi-modal graph fusion algorithm to realize fatigue relief, regulate depression and insomnia, and overall contains the following steps:

[0209] 1. Hardware: sandwich structure (aromatherapy micropump + semiconductor refrigerating sheet + comb-shaped dry electrode), realizing the integration of aromatherapy, temperature control and electroencephalogram acquisition

[0210] 2. Algorithm: time-domain differential convolution feature extraction + nonlinear graph fusion (multi-modal heterogeneous graph fusion of EEG features, aroma concentration and scalp temperature)

[0211] 3. Control: closed-loop control strategy based on reinforcement learning (state space: EEG features + temperature + odor concentration; action space: temperature control instruction / aromatherapy pulse; reward function fusion of physiological indicators)

[0212] Has the following beneficial effects:

[0213] Multi-modal perception and fusion are adopted: the system does not rely on a single electroencephalogram signal, but integrates data of electroencephalogram (physiological state), temperature (physical state) and aromatherapy (chemical stimulation). This integration provides a more comprehensive and three-dimensional user state portrait, overcoming the shortcomings of single signal interference and information fragmentation, greatly improving the accuracy of state recognition (such as relaxation, anxiety and sleep stage).

[0214] Nonlinear heterogeneous graph fusion model is adopted: this advanced algorithm can intelligently capture complex and nonlinear physiological and physical relationships between different modal data (such as the promotion of a certain aroma to alpha brain waves at a certain temperature). This is more revealing of deep state associations than simple linear weighted fusion, providing a solid basis for accurate regulation.

[0215] Adaptive closed-loop with reinforcement learning: The system is not a static, one-size-fits-all stimulator. It can adjust the strategy in real time according to the actual effect (reward value) after each intervention. This means that the system can continuously learn and adapt to the unique physiological response patterns of each user (for example, user A is sensitive to lavender, while user B may need different temperature settings), truly realizing the "the more you use, the smarter" personalized adaptive optimization, and the long-term effect is better.

[0216] It should be noted that the method of the embodiments of the present application can be executed by a single device, such as a computer or a server, etc. The method of the embodiments can also be applied to a distributed scenario, and be completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only execute one or more steps in the method of the embodiments of the present application, and the multiple devices will interact with each other to complete the method.

[0217] It should be noted that some embodiments of the present application have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order described above and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0218] Based on the same inventive concept, the present application also provides an electronic device corresponding to any of the above-mentioned embodiment methods, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to realize the adaptive closed-loop electroencephalogram regulation method based on multi-modal feedback of any one of the above-mentioned embodiments.

[0219] Figure 6 A more specific hardware structure schematic diagram of an electronic device provided by the present embodiment is shown, which can include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040 and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030 and the communication interface 1040 are connected to each other through the bus 1050 for communication within the device.

[0220] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing relevant programs to implement the technical solutions provided by the embodiments of the present specification.

[0221] The memory 1020 can be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the relevant program codes are saved in the memory 1020 and called and executed by the processor 1010.

[0222] The input / output interface 1030 is configured to connect input / output modules to implement information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input devices can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices can include a display, a speaker, a vibrator, an indicator light, etc.

[0223] The communication interface 1040 is configured to connect a communication module (not shown in the figure) to implement the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.) or through a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0224] The bus 1050 includes a channel for transmitting information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0225] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only include the components necessary to implement the solutions of the embodiments of the present specification, and does not have to include all the components shown in the figure.

[0226] The electronic device of the above-mentioned embodiments is used to implement the corresponding adaptive closed-loop electroencephalogram regulation method based on multi-modal feedback in any of the preceding embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0227] Based on the same inventive concept, corresponding to the method of any of the above-mentioned embodiments, the present application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the adaptive closed-loop electroencephalogram regulation method based on multi-modal feedback as described in any of the above embodiments.

[0228] The computer-readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0229] The computer instructions stored in the storage medium of the above-mentioned embodiments are used to cause the computer to execute the adaptive closed-loop electroencephalogram regulation method based on multi-modal feedback as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0230] It can be understood that before using the technical solutions of various embodiments in the present disclosure, the user will be informed of the type, use range, use scenario, etc. of the personal information involved in a proper manner, and the authorization of the user will be obtained.

[0231] For example, in response to receiving the user's active request, prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require the acquisition and use of the user's personal information. Thus, the user can voluntarily choose whether to provide personal information to the software or hardware such as electronic devices, application programs, servers or storage media that perform the technical solutions of the present disclosure according to the prompt information.

[0232] As an optional but non-limiting implementation manner, in response to accepting the active request of the user, the manner of sending the prompt information to the user may be, for example, a pop-up window manner, in which the prompt information may be presented in a text manner. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide the personal information to the electronic device.

[0233] It can be understood that the above notification and user authorization obtaining process is only illustrative, and does not limit the implementation manners of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation manners of the present disclosure.

[0234] It should be understood by those skilled in the art that the above discussion of any embodiment is only exemplary and is not intended to limit the scope of the present application to these examples; under the idea of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the embodiments of the present application as described above. In order to be brief, they are not provided in detail.

[0235] In addition, in order to simplify the description and discussion, and so as not to make the embodiments of the present application difficult to understand, the known power / ground connections of integrated circuit (IC) chips and other components can or can not be shown in the provided drawings. In addition, the devices can be shown in the form of block diagrams in order to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (i.e. these details should be fully within the understanding of those skilled in the art). Where specific details (e.g. circuits) are set forth in order to describe an exemplary embodiment of the present application, it will be apparent to those skilled in the art that the present application can be practiced without these specific details or with variations of these specific details. Therefore, these descriptions should be considered as illustrative rather than limiting.

[0236] Although the present application has been described in conjunction with specific embodiments thereof, many alternatives, modifications and variations will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g. dynamic RAM (DRAM)) can use the embodiments discussed.

[0237] Any omissions, modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present application shall be included in the protection scope of the present application.

Claims

1. A reinforcement learning closed-loop control system based on multi-modal feedback, characterized in that, The system comprises: a comb-shaped dry electrode array arranged on the forehead and temple regions of the head for collecting multi-channel electroencephalogram signals; a semiconductor refrigeration sheet co-located with the comb-shaped dry electrode for adjusting the temperature of the head; a temperature sensor for real-time monitoring of scalp temperature; an aromatherapy micro-pump device for storing and controllably releasing aromatherapy molecules; a signal acquisition and preprocessing module connected to the head-mounted device for filtering and denoising the collected electroencephalogram signals and standardizing the temperature signals; a multi-modal feature extraction and state recognition module for fusion analysis of the preprocessed electroencephalogram signals, temperature data, and aromatherapy concentration data based on a time-domain differential convolution algorithm and a nonlinear heterogeneous graph fusion model, and outputting user physiological state recognition results; a reinforcement learning closed-loop control module for generating aromatherapy release control instructions and temperature adjustment instructions based on a pre-set reward function and a policy network according to the physiological state recognition results, and sending them to the head-mounted device for execution; a user data management module for storing user historical data, personalized schemes, and scene mode libraries.

2. The reinforcement learning closed-loop control system based on multi-modal feedback according to claim 1, wherein, At least one electrode in the comb-shaped dry electrode array is a multiplex electrode that has both the functions of collecting electroencephalogram signals and conducting temperature adjustment.

3. The reinforcement learning closed-loop control system based on multi-modal feedback according to claim 1 or 2, wherein, The nonlinear heterogeneous graph fusion model defines electroencephalogram features, scalp temperature, and aromatherapy concentration as different node types in the heterogeneous graph and defines edges based on the physical and physiological relationships between nodes, and performs feature fusion through a graph neural network.

4. The reinforcement learning closed-loop control system based on multi-modal feedback according to claim 1, wherein, The time-domain differential convolution algorithm includes first-order differential processing of input electroencephalogram signals and temperature signals, and convolution operations using convolution kernels matched to the characteristics of each signal to extract dynamic features.

5. The reinforcement learning closed-loop control system based on multi-modal feedback of claim 1, wherein, The reinforcement learning closed-loop control module uses a deep deterministic policy gradient algorithm, with a state space including electroencephalogram frequency band power, temperature gradient, and aromatherapy concentration, an action space including aromatherapy pulse change and temperature change instructions, and a reward function that combines deep sleep promotion indicators, temperature steady-state indicators, and aromatherapy precision indicators.

6. The reinforcement learning closed-loop control system based on multi-modal feedback of claim 1, wherein, The multi-modal feedback-based reinforcement learning closed-loop regulation system further comprises a scene adaptation module for automatically calling corresponding stimulation mode libraries according to user identity or environmental information, with the stimulation mode libraries including a first mode for beauty relaxation, a second mode for traditional Chinese medicine therapy, and a third mode for psychological anxiety intervention.

7. A reinforcement learning closed-loop regulation method based on electroencephalogram-temperature-aroma multi-modal feedback, characterized in that, The method is applied to the multi-modal feedback-based reinforcement learning closed-loop regulation system as claimed in any of claims 1-6, and comprises: synchronously collecting electroencephalogram signals and head temperature data of a user through an integrated head-mounted device; preprocessing the collected electroencephalogram and temperature signals; extracting dynamic features of the signals using a time-domain differential convolution algorithm; inputting the extracted features and real-time aromatherapy concentration data into a nonlinear heterogeneous graph fusion model for multi-modal feature fusion and user physiological state recognition; inputting the recognized physiological state into a pre-trained reinforcement learning control model to generate aromatherapy release control instructions and / or temperature adjustment instructions; sending the instructions to the head-mounted device for execution to complete a regulation cycle; based on newly collected physiological data after execution, calculating a reward value and updating the policy of the reinforcement learning control model for closed-loop adaptive optimization.

8. The method of claim 7, wherein, The training process of the reinforcement learning control model comprises: generating continuous control actions through a policy network and evaluating and optimizing the actions through a value network with the goal of maximizing cumulative rewards, wherein a reward function is determined by a deep sleep electroencephalogram power change, a deviation of a scalp temperature from a target temperature, and a deviation of an essential oil concentration from a target concentration.

9. The method according to claim 7 or 8, characterized in that, Before generating the control instruction, the method further comprises: calling a stimulation mode library matched with a current user scenario to set initial parameters or constraint ranges of an action space of the reinforcement learning model.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by the processor, implements the steps of the method of any one of claims 7 to 9.

Images