Brain-computer interface exoskeleton multi-dimensional feedback adaptive control method and system

Abstract

The application relates to the fields of brain-computer interface, rehabilitation engineering and intelligent control technology, and provides a brain-computer interface exoskeleton multidimensional feedback adaptive control method and system. The method comprises the following steps: collecting multidimensional state information containing electroencephalogram features, exoskeleton physical states and current decoding intentions, and constructing a multidimensional state space; detecting an error-related potential automatically triggered by the brain due to the fact that the exoskeleton action is inconsistent with the user intention, and taking the error-related potential as an internal reward signal for correcting the multidimensional state information; inputting physical indexes including joint angles of the exoskeleton into the reinforcement learning intelligent agent, combining electroencephalogram signals and physical end feedback, and reversely optimizing decoding parameters of the electroencephalogram decoder; and optimizing decoding parameters corresponding to the multidimensional state information according to real-time performances of each user, generating a dynamically evolved control logic, and realizing adaptive and personalized control.

Description

Technical Field

[0001] This application relates to the fields of brain-computer interface, rehabilitation engineering and intelligent control technology, and in particular to a multidimensional feedback adaptive control method and system for a brain-computer interface exoskeleton. Background Technology

[0002] The application of brain-computer interface (BCI) exoskeleton systems in rehabilitation has gradually become widespread, but their core control technology still faces significant bottlenecks. Existing solutions primarily rely on fixed-parameter EEG decoders, which cannot cope with the non-stationary characteristic drift of EEG signals caused by user fatigue, attention fluctuations, or the recovery of neural plasticity. Recognition accuracy decreases significantly with long-term use. Furthermore, traditional systems rely on "true intention labels" to correct models, but in actual rehabilitation, there is a lack of real-time label sources, making it difficult for the exoskeleton to self-correct erroneous movements without interrupting training. In addition, existing technologies are mostly one-way control modes, driving exoskeleton movement solely through EEG signals, ignoring the feedback influence of the exoskeleton's execution state (such as movement smoothness and interaction torque) on human neural signals, resulting in an incomplete human-computer interaction loop. More importantly, the EEG characteristics of each patient vary greatly, and traditional fixed control modes require frequent manual recalibration, leading to high clinical parameter tuning costs and difficulty in supporting home-based autonomous rehabilitation.

[0003] Therefore, a method is urgently needed to solve at least one of the above problems. Summary of the Invention

[0004] This application provides a multidimensional feedback adaptive control method and system for a brain-computer interface exoskeleton, which aims to solve the following problem.

[0005] In a first aspect, embodiments of this application provide a multidimensional feedback adaptive control method for a brain-computer interface exoskeleton, the method comprising: Collect multidimensional state information including EEG features, exoskeleton physical state, and current decoding intent, and construct a multidimensional state space based on the multidimensional state information; The hyperparameters of the EEG decoder are adjusted by using a reinforcement learning agent, so as to model the parameter optimization process of the EEG decoder as a decision-making process; the action vectors corresponding to the multi-dimensional state information output by the reinforcement learning agent are used to correct the hyperparameter set of the EEG decoder. The multidimensional state information is used to detect the error-related potentials automatically triggered by the brain due to the inconsistency between the exoskeleton's movements and the user's intentions. These potentials are then used as internal reward signals to correct the multidimensional state information. Physical indicators, including the joint angles of the exoskeleton, are input into the reinforcement learning agent. Combined with EEG signals and physical feedback, the decoding parameters of the EEG decoder are optimized in reverse. Based on each user's real-time performance, the decoding parameters corresponding to the multi-dimensional state information are optimized to generate dynamically evolving control logic, thereby achieving adaptive personalized control.

[0006] In some embodiments, the acquisition of multidimensional state information including EEG features, exoskeleton physical state, and current decoding intent includes: extracting power spectral density features and feature vectors based on common space pattern spatial filtering of EEG signals in real time using an EEG acquisition device as EEG features; acquiring the angles, angular velocities, human-computer interaction torques, instantaneous power consumption change rates of motors, and jerk terms of each joint of the exoskeleton using joint angle sensors, angular velocity sensors, torque sensors, and motor power consumption monitoring modules as the exoskeleton physical state; and calculating the intent classification probability distribution and its confidence level output by the EEG decoder as the current decoding intent.

[0007] In some embodiments, constructing a multidimensional state space based on the multidimensional state information includes: taking EEG features, exoskeleton physical state, and current decoding intent as three components of the multidimensional state space, and combining them to form a multidimensional feature vector containing a time series; wherein the EEG feature component is used to characterize the user's neural signal state, the exoskeleton physical state component is used to characterize the human-machine coupling dynamics, and the current decoding intent component is used to characterize the reliability assessment of the current control command.

[0008] In some embodiments, the step of using a reinforcement learning agent to adjust the hyperparameters of the EEG decoder to model the parameter optimization process of the EEG decoder as a decision process includes: modeling the parameter optimization process of the EEG decoder as a Markov decision process, defining a quadruple model including a state space, an action space, transition probabilities, and a reward function, and selecting an adjustment strategy for the hyperparameters of the EEG decoder based on the input of the multidimensional state space by the reinforcement learning agent, with the reward signal from the environmental feedback as the optimization target.

[0009] In some embodiments, the step of correcting the hyperparameter set inside the EEG decoder by outputting action vectors corresponding to multidimensional state information through a reinforcement learning agent includes: using the action vectors output by the reinforcement learning agent as the online correction amount for the hyperparameter set inside the EEG decoder, and determining whether to generate a valid EEG control command based on the corrected decision threshold; wherein the decision threshold is dynamically adjusted according to the level of environmental noise and the clarity of the user's intent, increasing the threshold when the environmental noise is high to prevent accidental touches, and decreasing the threshold when the user's intent is clear to enhance the response.

[0010] In some embodiments, the step of detecting error-related potentials automatically triggered by the brain due to discrepancies between exoskeleton movements and user intentions based on the multidimensional state information, and using these as internal reward signals to correct the multidimensional state information, includes: analyzing the characteristic waveforms of error-related potentials in the electroencephalogram (EEG) signal, calculating the deviation between the intention classification probability distribution output by the decoder and the actual action feedback, and determining that an error-related potential is triggered when the deviation exceeds a preset threshold, generating a physiological feedback reward signal to update the evaluation mechanism of the reinforcement learning agent.

[0011] In some embodiments, the step of inputting physical indicators, including the joint angles of the exoskeleton, into the reinforcement learning agent, and combining EEG signals and physical feedback to backpropagate and optimize the decoding parameters of the EEG decoder, includes: inputting physical indicators, including exoskeleton joint angles, angular velocity, jerk terms, human-computer interaction torque, and motor power consumption, into the reward function of the reinforcement learning agent to calculate physical smoothness rewards and energy consumption rewards; and optimizing the decoding parameters of the EEG decoder through a backpropagation algorithm to ensure that the exoskeleton motion smoothness and energy output stability meet preset standards.

[0012] In some embodiments, the step of optimizing the decoding parameters corresponding to the multi-dimensional state information based on the real-time performance of each user to generate dynamically evolving control logic includes: using a linear decay mechanism to dynamically adjust the random exploration probability and action noise intensity of the reinforcement learning agent, so that the system can smoothly transition from the extensive exploration stage to the precise adaptation stage; and using a soft update mechanism to slowly synchronize the target network parameters to avoid the exoskeleton from running out of control due to drastic changes in network weights.

[0013] In some embodiments, the method further includes: introducing time-dependent noise into the action vector output by the reinforcement learning agent to avoid mechanical shaking of the exoskeleton caused by the adjustment of EEG decoder parameters; wherein the noise intensity gradually decreases with training time to ensure the refinement of subsequent parameter adjustment and the stability of the control process.

[0014] Secondly, this application provides a multi-dimensional feedback adaptive control system for a brain-computer interface exoskeleton, the system comprising: The information acquisition unit is used to acquire multi-dimensional state information including EEG characteristics, exoskeleton physical state and current decoding intent, and to construct a multi-dimensional state space based on the multi-dimensional state information. The parameter correction unit is used to adjust the hyperparameters of the EEG decoder using a reinforcement learning agent, so as to model the parameter optimization process of the EEG decoder as a decision-making process; and to correct the hyperparameter set inside the EEG decoder by outputting action vectors corresponding to multi-dimensional state information through the reinforcement learning agent. The parameter optimization unit is used to detect the error-related potentials automatically triggered by the brain due to the inconsistency between the exoskeleton's movements and the user's intentions based on the multidimensional state information, and use them as internal reward signals to correct the multidimensional state information; it inputs physical indicators, including the joint angles of the exoskeleton, into the reinforcement learning agent, and combines EEG signals and physical feedback to back-optimize the decoding parameters of the EEG decoder. The control completion unit is used to optimize the decoding parameters corresponding to the multi-dimensional state information based on the real-time performance of each user, generate dynamically evolving control logic, and realize adaptive personalized control.

[0015] This application overcomes the challenge of non-stationarity in EEG signals, achieving long-term, precise control. Utilizing an online update mechanism of reinforcement learning agents, this invention can track and compensate for feature "drift" in user EEG signals caused by fatigue, attention fluctuations, or neural recovery in real time. Compared to traditional fixed-parameter decoders, the system dynamically adjusts classification weights and thresholds to ensure that recognition accuracy does not decay over time during long-term rehabilitation training, significantly improving the system's robustness. This invention constructs a physiological-physical fusion state observation space, enabling the exoskeleton to simultaneously "sense" the user's neural intentions and the mechanical execution state. The introduction of a physical smoothing reward based on jerk (Jerk) terms reduces the jitter (Jerk value) of the mechanical end-effector trajectory, effectively suppressing mechanical vibration or human-machine aggression caused by decoding fluctuations, providing patients with a more natural and smoother power assistance experience.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart illustrating the steps of a multidimensional feedback adaptive control method for a brain-computer interface exoskeleton according to an embodiment of this application; Figure 2 This is a schematic diagram of a human-machine feedback closed-loop principle provided in one embodiment of this application; Figure 3 An embodiment of this application provides an internal architecture diagram of a reinforcement learning agent; Figure 4 This is a schematic block diagram of a multidimensional feedback adaptive control system for a brain-computer interface exoskeleton provided in one embodiment of this application; Figure 5 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0022] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0023] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] The application of brain-computer interface (BCI) exoskeleton systems in rehabilitation has gradually become widespread, but their core control technology still faces significant bottlenecks. Existing solutions primarily rely on fixed-parameter EEG decoders, which cannot cope with the non-stationary characteristic drift of EEG signals caused by user fatigue, attention fluctuations, or the recovery of neural plasticity. Recognition accuracy decreases significantly with long-term use. Furthermore, traditional systems rely on "true intention labels" to correct models, but in actual rehabilitation, there is a lack of real-time label sources, making it difficult for the exoskeleton to self-correct erroneous movements without interrupting training. In addition, existing technologies are mostly one-way control modes, driving exoskeleton movement solely through EEG signals, ignoring the feedback influence of the exoskeleton's execution state (such as movement smoothness and interaction torque) on human neural signals, resulting in an incomplete human-computer interaction loop. More importantly, the EEG characteristics of each patient vary greatly, and traditional fixed control modes require frequent manual recalibration, leading to high clinical parameter tuning costs and difficulty in supporting home-based autonomous rehabilitation.

[0026] Therefore, a method is urgently needed to solve at least one of the above problems.

[0027] To solve the above problem, please refer to Figure 1 This application provides a multi-dimensional feedback adaptive control method for a brain-computer interface exoskeleton, applied to a computer device. The computer device can be deployed on a single server or a server cluster. It can also be deployed on a handheld terminal, laptop, wearable device, or robot, etc. It should be noted that all information involved in the method provided in this application is extracted with the authorization of the relevant user and in accordance with relevant regulations, and will not infringe on user privacy.

[0028] The provided brain-computer interface exoskeleton multidimensional feedback adaptive control method includes steps S101 to S104. Details are as follows: Step S101. Collect multidimensional state information including EEG features, exoskeleton physical state and current decoding intent, and construct a multidimensional state space based on the multidimensional state information.

[0029] Specifically, this step is the perceptual foundation of the entire adaptive control method. Its core is to break through the limitations of traditional solutions that rely solely on a single EEG signal for unidirectional perception, and to construct a three-dimensional state observation system integrating physiological, physical, and decoding results. This provides a complete, real-time, and synchronous human-machine coupling system state basis for the decision optimization of reinforcement learning agents, and solves the problem of incomplete human-machine interaction closed loop caused by incomplete state perception in existing technologies.

[0030] Please refer to Figure 2 , Figure 2This paper visually demonstrates the physiological-physical dual closed-loop human-computer interaction control logic constructed in this application. The user's EEG signals and the physical state of the exoskeleton are jointly input into the multi-dimensional feedback adaptive controller. The controller dynamically adjusts the hyperparameters of the EEG decoder through a reinforcement learning agent to generate exoskeleton control commands. After the exoskeleton performs an action, the resulting error-related potentials and motion smoothness are fed back to the reinforcement learning agent, forming a complete self-optimizing closed loop.

[0031] This step synchronizes and fuses three core types of information—the user's neurophysiological state, the exoskeleton's mechanical execution state, and the decoder's current output state—to generate a standardized system state vector that can be directly input into the reinforcement learning agent. The system state vectors at all times together constitute a complete multi-dimensional state space, fully covering all core variables that affect the control effect of the brain-computer interface exoskeleton, thus avoiding biased optimization decisions due to missing state information.

[0032] The acquisition and quantification of EEG feature information involves real-time acquisition of the user's scalp EEG signals using an authorized EEG acquisition device. After preprocessing the raw EEG signals, such as denoising, filtering, and artifact removal, effective EEG features are extracted. These features include power spectral density features of specific frequency bands, energy features after spatial filtering, and variance features of the EEG signals. Simultaneously, error-related potential feature fragments for subsequent error correction are extracted, thus completing the standardized quantification of EEG feature information.

[0033] The acquisition and quantification of exoskeleton physical state information is achieved through sensors deployed on the exoskeleton, which collect real-time, multi-dimensional physical data of the exoskeleton's operation. Specifically, this includes real-time angle and angular velocity data of each joint of the exoskeleton, interaction torque and physical resistance values ​​between the user and the exoskeleton collected by the human-computer interaction torque sensor, and instantaneous power consumption data of the exoskeleton drive motor. At the same time, higher-order derivative processing is performed on the joint angle data to obtain jerk data used to characterize motion tremors, thus completing the standardization and quantification of the exoskeleton's physical state information.

[0034] The acquisition and quantification of current decoding intent information is achieved by using an EEG decoder to preliminarily decode the EEG features at the current moment, obtaining the current motion intent classification result and the confidence probability distribution corresponding to the classification result, thus completing the standardized quantification of the decoded intent information.

[0035] The final construction of the multidimensional state space involves synchronizing and fusing the EEG feature information, exoskeleton physical state information, and current decoding intent information collected at the same time with timestamps to generate the system state vector at the corresponding time. The system state vectors at all times together constitute a complete multidimensional state space, which serves as the sole input basis for the subsequent reinforcement learning agent.

[0036] Step S102. Use a reinforcement learning agent to adjust the hyperparameters of the EEG decoder, so as to model the parameter optimization process of the EEG decoder as a decision-making process; use the action vector corresponding to the multi-dimensional state information output by the reinforcement learning agent to correct the hyperparameter set inside the EEG decoder.

[0037] Specifically, this step is the core decision-making stage of the entire method. The core is to transform the traditional fixed-parameter EEG decoding process into an adaptive decision-making process that can be dynamically optimized online, solving the problem of long-term decoding accuracy decay caused by the drift of EEG features with the user's state. At the same time, it innovatively limits the output of the intelligent agent to the hyperparameter adjustment of the decoder, rather than the direct driving commands of the exoskeleton, thereby avoiding the risk of mechanical loss of control caused by drastic parameter adjustments from the root, and balancing the flexibility of optimization with the safety of use.

[0038] Please refer to Figure 3 , Figure 3 This paper demonstrates the internal architecture of the Deep Deterministic Policy Gradient (DDPG) reinforcement learning agent employed in this application. The agent receives a multi-dimensional state vector and a composite reward value as input, outputs the hyperparameters of the EEG decoder through the Actor network to regulate actions, evaluates the long-term value of actions through the Critic network, and utilizes an experience replay pool and a soft update mechanism to achieve stable online learning, avoiding exoskeleton malfunctions caused by drastic parameter jumps.

[0039] This step models the parameter optimization process of the EEG decoder as a Markov decision process, using the multidimensional state space constructed by S101 as the decision basis and the hyperparameter set of the EEG decoder as the optimization object. The online smooth correction of the decoder hyperparameters is achieved through the action vector output by the reinforcement learning agent, enabling the decoder to adapt to the dynamic changes of the user's EEG signal in real time, thereby improving the accuracy and stability of exoskeleton control from the root of decoding.

[0040] The decision modeling of the parameter optimization process is carried out by modeling the parameter optimization process of the EEG decoder as a Markov decision process. This process includes four core elements: the state space adopts a multi-dimensional state space constructed by S101, the action space is defined as the adjustment amount of the EEG decoder hyperparameters, the state transition rule is defined as the change law of the system state after the parameter adjustment action is executed, and the reward mechanism adopts the composite reward rule constructed by S103, thus completing the standardized modeling of the entire optimization process.

[0041] The construction and initialization of the reinforcement learning agent involves building a reinforcement learning agent that includes a policy network, a value network, and a target network. When the system starts up or a new user joins, two types of initialization operations are completed: first, the network parameters of the agent, such as network weights, learning rate, discount factor, soft update coefficient, and training batch size, are initialized; second, the initial hyperparameters of the EEG decoder are set, including the initial intent classification decision probability threshold, the initial EEG feature channel weight ratio, the initial feature extraction algorithm coefficients, and the learning rate for parameter updates.

[0042] The generation of hyperparameter-adjusted actions involves a reinforcement learning agent receiving the current system state vector generated by S101 and outputting the corresponding action vector through a policy network. This action vector is solely a dynamic adjustment of the EEG decoder's hyperparameters, specifically comprising three parts: first, the adjustment of the intent classification decision probability threshold, used to balance the sensitivity and accuracy of decoding and recognition; second, the incremental allocation of EEG feature channel weights, used to suppress signal channels with high noise interference and enhance the feature channels corresponding to the user's intention map; and third, the coefficient correction values ​​of the EEG feature extraction algorithm, used to optimize the extraction effect of EEG features.

[0043] The online update of EEG decoder hyperparameters is achieved by smoothly updating the current hyperparameter set of the EEG decoder based on the hyperparameter adjustment amount output by the reinforcement learning agent and combined with the preset parameter update learning rate, resulting in an optimized hyperparameter set. The updated hyperparameters directly affect the EEG signal decoding process at the next moment, realizing online adaptive adjustment of decoder parameters.

[0044] To balance the stability and exploratory nature of online learning for intelligent agents, time-dependent smooth noise is introduced into the action output of the policy network, while a greedy coefficient that decays with the training process is set. In the early stage of system operation, the optimal parameter range is found by exploring with a higher probability, and the exploration probability is gradually reduced in the later stage of operation to achieve a smooth transition from extensive exploration to precise adaptation, avoiding exoskeleton shaking caused by drastic parameter adjustments.

[0045] Step S103. Detect the error-related potentials automatically triggered by the brain due to the inconsistency between the exoskeleton's movements and the user's intentions based on the multidimensional state information, and use them as internal reward signals to correct the multidimensional state information; input physical indicators including the joint angles of the exoskeleton into the reinforcement learning agent, and combine the EEG signals and physical feedback to reverse-optimize the decoding parameters of the EEG decoder.

[0046] Specifically, this step is the core of the closed-loop feedback of the entire method. The core is to build a dual closed-loop composite reward mechanism of physiological feedback and physical feedback. On the one hand, it solves the pain points of not being able to obtain real-time true intention labels in rehabilitation scenarios and the system not being able to self-correct online. On the other hand, it solves the problems of traditional one-way control ignoring the influence of mechanical execution state on neural signals and the incomplete human-computer interaction closed loop. By using dual feedback signals to guide the optimization of decoding parameters in reverse, the system can achieve online self-evolution.

[0047] This step innovatively uses the brain's endogenous error-related potentials as physiological feedback rewards in unlabeled scenarios, enabling automatic identification and punishment of decoding errors without manual labeling. At the same time, the physical execution state of the exoskeleton is integrated into the reward mechanism and state input, constructing a complete human-computer interaction closed loop. By fusing three types of signals—physiological feedback, physical smoothing feedback, and task efficiency feedback—a composite reward value is generated, serving as the sole guiding signal for the optimization direction of the reinforcement learning agent, thus achieving closed-loop reverse optimization of decoding parameters.

[0048] The physiological feedback reward generation based on error-related potentials (ERPs) monitors error-related potentials in the user's EEG signals in real time. When the exoskeleton performs an action that does not match the user's true intention, the user's brain automatically triggers this potential. When the system detects this potential, it determines that the current decoding result is incorrect and generates a corresponding negative penalty reward. When the exoskeleton completes the action and no potential is detected, it determines that the current decoding result is correct and generates a corresponding positive reward. When no action is performed, a zero reward is generated, ultimately yielding a physiological feedback reward value without the need for additional manual labeling of intent.

[0049] The physical smoothing reward generation based on the physical state of the exoskeleton calculates the physical smoothing reward value based on the physical state information of the exoskeleton collected by S101. Specifically, it is calculated by comprehensively considering the acceleration data of the exoskeleton joints, the deviation data between real-time interactive torque and average torque, and the rate of change of instantaneous power consumption of the motor. The smoother the exoskeleton movement, the smaller the human-machine interaction resistance, and the more stable the energy output, the higher the corresponding physical smoothing reward value, and vice versa. This is used to guide the intelligent agent to suppress mechanical vibration and human-machine resistance caused by decoding fluctuations.

[0050] Task efficiency reward generation based on rehabilitation task execution: Based on the preset rehabilitation training tasks, the progress and accuracy of the exoskeleton in performing the tasks are tracked in real time; the task completion progress is calculated by the remaining distance to the rehabilitation target position and the time spent on the task; at the same time, the trajectory tracking error is calculated by the deviation between the actual joint angle of the exoskeleton and the standard angle of the preset rehabilitation trajectory; and the task efficiency reward value is generated by combining the progress and error data to guide the agent to complete the core goal of rehabilitation training while ensuring accurate decoding.

[0051] The aforementioned physiological feedback rewards, physical smoothing rewards, and task efficiency rewards are fused according to preset weight coefficients to generate a final composite reward value. This composite reward value serves as the core guiding signal for the optimization direction of the reinforcement learning agent. Combined with the multidimensional state information of S101 and the parameter adjustment actions of S102, it constitutes the optimization basis of the agent, realizing a closed-loop logic of reverse optimization of EEG decoder parameters through physiological and physical dual feedback.

[0052] The system state information at the current moment, parameter adjustment actions, composite reward value, and system state information at the next moment after the action are executed are combined to form an interaction trajectory sample, which is stored in the experience replay pool. When the number of samples in the experience replay pool reaches the preset batch processing threshold, the system starts the background asynchronous update process, randomly selects batches of samples from the experience replay pool, updates the value network by minimizing value loss, updates the policy network by using the policy gradient algorithm, and at the same time uses a soft update mechanism to slowly synchronize the parameters of the target network to avoid the exoskeleton from running out of control due to drastic changes in network weights, thus completing the continuous iterative optimization of the agent.

[0053] Step S104. Based on the real-time performance of each user, optimize the decoding parameters corresponding to the multi-dimensional state information, generate dynamically evolving control logic, and realize adaptive personalized control.

[0054] Specifically, this step is the core of the personalized implementation of the entire method. The core is to solve the pain points of high cost of personalized adaptation, heavy burden of manual parameter adjustment, and inability to adapt to the user's rehabilitation progress in the traditional fixed control mode. It achieves dynamic personalized adaptation for different users and different rehabilitation stages, significantly reduces the cost of clinical intervention, and provides technical support for users' home-based self-rehabilitation.

[0055] This step leverages the online optimization capabilities of reinforcement learning, taking the user's real-time physiological performance and rehabilitation training effects as the core basis. It automatically searches for the optimal decoding parameter space for each user and generates a unique dynamic evolution control logic. This eliminates the need for frequent manual recalibration and parameter tuning, adapting to the differences in EEG characteristics among different users, as well as the changes in neural signals at different rehabilitation stages and under different physical conditions for the same user, achieving full-cycle adaptive personalized control.

[0056] By continuously collecting real-time performance data of users during rehabilitation training, including the triggering frequency of error-related potentials, the accuracy of EEG decoding, the smoothness of exoskeleton movement, the success rate of rehabilitation task completion, the error data of trajectory tracking, as well as the EEG characteristic changes corresponding to the user's rehabilitation training duration and fatigue state, the user's real-time performance is standardized and quantified.

[0057] By using reinforcement learning agents with real-time user performance data as the core basis and maximizing composite reward value as the optimization goal, the system automatically searches online for the optimal EEG decoder parameter range based on the user's EEG characteristics, motor ability, and rehabilitation progress. It continuously adjusts hyperparameters such as the decoder's classification threshold, feature channel weights, and feature extraction coefficients to generate a unique decoding parameter set for each user, replacing the traditional fixed general parameters.

[0058] By combining the changes in EEG characteristics brought about by the recovery of the user's neuroplasticity, as well as the fatigue and attention fluctuations during the rehabilitation training process, the system continuously updates the decoding parameters dynamically, so that the control logic evolves in sync with the user's physical state and rehabilitation progress. This eliminates the need for manual recalibration and parameter adjustment, and fundamentally solves the performance degradation problem caused by EEG characteristic drift during long-term use.

[0059] During personalized parameter optimization, safety boundaries for parameter adjustment are set to avoid exoskeleton malfunctions caused by adjusting parameters beyond the range. For new users, the system quickly completes initial adaptation through preset initial parameters, significantly shortening the initial offline calibration time and achieving a "plug-and-play" effect. During long-term rehabilitation, the system continuously and automatically optimizes the control logic, eliminating the need for frequent intervention and parameter adjustment by clinicians, reducing clinical usage costs, and providing stable technical support for users' self-rehabilitation at home.

[0060] In some embodiments, the acquisition of multidimensional state information including EEG features, exoskeleton physical state, and current decoding intent includes: extracting power spectral density features and feature vectors based on common space pattern spatial filtering of EEG signals in real time using an EEG acquisition device as EEG features; acquiring the angles, angular velocities, human-computer interaction torques, instantaneous power consumption change rates of motors, and jerk terms of each joint of the exoskeleton using joint angle sensors, angular velocity sensors, torque sensors, and motor power consumption monitoring modules as the exoskeleton physical state; and calculating the intent classification probability distribution and its confidence level output by the EEG decoder as the current decoding intent.

[0061] This embodiment provides a standardized, multi-dimensional, and time-synchronized data source for the construction of a multi-dimensional state space, which is the perception foundation of the entire adaptive control method. It solves the problem of the one-sided decision-making basis caused by existing technologies relying on only a single EEG signal and incomplete perception dimensions. It fully covers three core control-influencing variables: user neurophysiological state, exoskeleton mechanical execution state, and decoder output state.

[0062] This embodiment formulates corresponding acquisition and quantification rules for the three core components of multidimensional state information: standardized EEG features are extracted through EEG acquisition equipment, physical operation data in all dimensions are collected through multiple types of exoskeleton sensors, and the current decoding intent is quantified through the output of the EEG decoder. All three types of data are preprocessed and standardized, providing input data in a unified format for subsequent state space construction.

[0063] EEG feature acquisition involves real-time acquisition of raw EEG signals from the user's scalp using an authorized EEG acquisition device. The raw signals are first subjected to standardized preprocessing, including denoising, filtering, and artifact removal. Then, power spectral density features of specific frequency bands and energy feature vectors based on common spatial pattern spatial filtering are extracted from the preprocessed signals. Both types of features are used together as standardized EEG feature data.

[0064] The exoskeleton's physical state is acquired by deploying joint angle sensors, angular velocity sensors, human-machine interaction torque sensors, and a motor power consumption monitoring module at corresponding locations on the exoskeleton. Real-time angle and angular velocity data of each joint on the exoskeleton are obtained through the angle and angular velocity sensors, and high-order derivatives are performed on the joint angle data to obtain an acceleration term characterizing motion tremors. Real-time human-machine interaction torque data between the user and the exoskeleton is collected through the torque sensors. The instantaneous rate of change of power consumption of the exoskeleton's drive motors is obtained through the motor power consumption monitoring module. All of the above data are combined to form standardized exoskeleton physical state data.

[0065] The current decoding intent acquisition uses an EEG decoder to pre-decode the preprocessed EEG signal at the current moment, outputs the corresponding motor intent classification result, and calculates the probability distribution and confidence value corresponding to the classification result. The intent classification result, probability distribution and confidence value are used together as standardized current decoding intent data.

[0066] In some embodiments, constructing a multidimensional state space based on the multidimensional state information includes: taking EEG features, exoskeleton physical state, and current decoding intent as three components of the multidimensional state space, and combining them to form a multidimensional feature vector containing a time series; wherein the EEG feature component is used to characterize the user's neural signal state, the exoskeleton physical state component is used to characterize the human-machine coupling dynamics, and the current decoding intent component is used to characterize the reliability assessment of the current control command.

[0067] This embodiment focuses on the practical application of the collected data. Its core is to integrate the fragmented three types of collected data into a standardized state vector that can be directly input into the reinforcement learning agent, thereby constructing a complete multi-dimensional state space. This solves the problem of asynchronous state information in existing technologies, which cannot be directly used for reinforcement learning decisions, and provides a comprehensive and unified input basis for the subsequent parameter optimization decisions of the agent.

[0068] In this embodiment, the collected EEG features, exoskeleton physical state, and current decoding intent are respectively used as three independent components of the multidimensional state space. Through time synchronization and standardization, they are spliced ​​together to form a multidimensional feature vector with time series. At the same time, the representational role of each component is clarified to ensure that the state space can fully reflect the real-time operating state of the human-machine coupling system, providing comprehensive support for reinforcement learning decision-making.

[0069] Data time synchronization processing aligns EEG features, exoskeleton physical state, and current decoding intent data collected at the same time using a unified timestamp, eliminating the time difference between data collected by different devices and modules and ensuring the temporal consistency of the three sets of data.

[0070] The system uses time-synchronized EEG features as the first component of a multidimensional state space to represent the user's neural signal state in real time; the exoskeleton physical state as the second component of a multidimensional state space to represent the limb dynamics features during human-machine coupling in real time; and the current decoded intent as the third component of a multidimensional state space to represent the reliability assessment result of the current control command in real time. The three components are then standardized and concatenated in a preset order to generate a multidimensional feature vector at the corresponding time.

[0071] By integrating the multidimensional feature vectors generated at each moment throughout the entire system's operation in a time sequence, a complete multidimensional state space is formed. This state space will serve as the sole input for subsequent reinforcement learning agent decisions.

[0072] In some embodiments, the step of using a reinforcement learning agent to adjust the hyperparameters of the EEG decoder to model the parameter optimization process of the EEG decoder as a decision process includes: modeling the parameter optimization process of the EEG decoder as a Markov decision process, defining a quadruple model including a state space, an action space, transition probabilities, and a reward function, and selecting an adjustment strategy for the hyperparameters of the EEG decoder based on the input of the multidimensional state space by the reinforcement learning agent, with the reward signal from the environmental feedback as the optimization target.

[0073] This embodiment is the core modeling step of the entire adaptive control method. The core is to transform the traditional fixed-parameter EEG decoding process into a Markov decision process that can be dynamically optimized online. This provides a standardized mathematical model framework for parameter optimization of reinforcement learning agents and solves the core pain points of existing fixed-parameter decoders being unable to adapt to the non-stationarity of EEG signals and the decline in accuracy over long-term use.

[0074] This embodiment models the parameter optimization process of the EEG decoder as a Markov decision process, defining a four-tuple model that includes state space, action space, transition probability, and reward function, and clarifying the definition and boundary of each core element in the model; at the same time, taking the maximization of the reward signal from environmental feedback as the optimization objective, it enables the reinforcement learning agent to autonomously select the optimal EEG decoder hyperparameter adjustment strategy based on the input of the multi-dimensional state space, thereby achieving online adaptive optimization of the decoding parameters.

[0075] The Markov decision process quadruple model is constructed by modeling the parameter optimization process of the EEG decoder as a Markov decision process quadruple model containing four core elements: state space, action space, transition probability, and reward function.

[0076] The core elements of the quadruple model are defined as follows: the constructed multidimensional state space is defined as the model's state space; the adjustment range of all optimizable hyperparameters of the EEG decoder is defined as the model's action space; the probability distribution of the human-machine coupling system transitioning from the current state to the next state after performing parameter adjustment actions is defined as the model's transition probability; and the composite reward signal generated by fusing subsequent physiological feedback, physical feedback, and task efficiency feedback is defined as the model's reward function. The reinforcement learning agent receives input data from the multidimensional state space at the current moment. Using the quadruple model as a framework and maximizing the reward signal from environmental feedback as the optimization objective, it autonomously selects and outputs adjustment strategies for the EEG decoder hyperparameters, achieving standardized and iterative decision-making management of the decoding parameter optimization process.

[0077] In some embodiments, the step of correcting the hyperparameter set inside the EEG decoder by outputting action vectors corresponding to multidimensional state information through a reinforcement learning agent includes: using the action vectors output by the reinforcement learning agent as the online correction amount for the hyperparameter set inside the EEG decoder, and determining whether to generate a valid EEG control command based on the corrected decision threshold; wherein the decision threshold is dynamically adjusted according to the level of environmental noise and the clarity of the user's intent, increasing the threshold when the environmental noise is high to prevent accidental touches, and decreasing the threshold when the user's intent is clear to enhance the response.

[0078] This embodiment focuses on the core action execution stage of the reinforcement learning agent. The key is to clearly define the action output boundary of the agent, limiting the agent's output to the correction amount of the EEG decoder hyperparameters, rather than the direct driving commands of the exoskeleton. This avoids the risk of mechanical loss of control caused by drastic parameter adjustments from the root. At the same time, by dynamically adjusting the decision threshold, the sensitivity and accuracy of decoding and recognition are balanced, solving the problems of false touches in noisy environments and untimely response when the user's intention is clear.

[0079] In this embodiment, the action vector output by the reinforcement learning agent is strictly limited to the online correction amount of the hyperparameter set inside the EEG decoder, rather than the underlying execution torque of the exoskeleton motor. The core optimization object is the intention classification decision threshold of the EEG decoder. At the same time, the decision threshold is dynamically and adaptively adjusted according to the environmental noise level and the clarity of the user's intention to ensure that the decoding process takes into account both safety and response speed.

[0080] After receiving the multidimensional state space data at the current moment, the reinforcement learning agent outputs the corresponding action vector. This action vector is only an online correction of the hyperparameter set inside the EEG decoder. Specifically, it includes three categories: adjustment of the intent classification decision probability threshold, allocation increment of EEG feature channel weights, and coefficient correction value of the EEG feature extraction algorithm. It does not directly generate drive control commands for the exoskeleton.

[0081] Based on the action vector output by the agent, and combined with the preset parameter update learning rate, the current hyperparameter set of the EEG decoder is smoothly updated to obtain the corrected hyperparameter set, in which the core correction object is the decision threshold for intent classification.

[0082] Based on the corrected decision threshold, the intent classification probability output by the EEG decoder is determined. When the intent classification probability is greater than or equal to the corrected decision threshold, a valid target EEG control command is generated; when the intent classification probability is less than the corrected decision threshold, a standby command is generated and the exoskeleton action is not triggered.

[0083] When high levels of environmental noise or significant EEG interference are detected, the decision threshold is automatically increased to prevent accidental triggering of exoskeleton movements. When the user's intent is clear and exoskeleton assistance is urgently needed, the decision threshold is automatically decreased to enhance the system's response sensitivity and achieve a dynamic balance between sensitivity and accuracy.

[0084] In some embodiments, the step of detecting error-related potentials automatically triggered by the brain due to discrepancies between exoskeleton movements and user intentions based on the multidimensional state information, and using these as internal reward signals to correct the multidimensional state information, includes: analyzing the characteristic waveforms of error-related potentials in the electroencephalogram (EEG) signal, calculating the deviation between the intention classification probability distribution output by the decoder and the actual action feedback, and determining that an error-related potential is triggered when the deviation exceeds a preset threshold, generating a physiological feedback reward signal to update the evaluation mechanism of the reinforcement learning agent.

[0085] This embodiment is the core of the physiological feedback closed loop of the entire method. Its core is to solve the industry pain points of not being able to obtain real-time and true intention labels in rehabilitation scenarios and the inability of traditional systems to complete self-correction without interrupting training. It breaks through the dependence of supervised learning on manual labeling and realizes the system's online autonomous error correction and parameter optimization.

[0086] This embodiment analyzes the error-related potential characteristic waveforms in EEG signals to identify the deviation between exoskeleton movements and the user's true intentions. The detection results of error-related potentials are used as internal physiological feedback reward signals and integrated into the evaluation mechanism of reinforcement learning agents. This enables automatic identification, punishment, and parameter correction of decoding errors without manual annotation.

[0087] During system operation, the system continuously performs feature analysis on the collected EEG signals, focusing on identifying the characteristic waveforms of error-related potentials automatically triggered by the brain due to the inconsistency between the exoskeleton's movements and the user's intentions.

[0088] By calculating the probability distribution of intent classification output by the EEG decoder and the matching degree with the actual action feedback of the exoskeleton, when the deviation between the two exceeds a preset threshold and the characteristic waveform of error-related potential is detected simultaneously, it is determined that the current decoding result is incorrect and the action performed by the exoskeleton does not match the user's true intent.

[0089] When a decoding error is detected or an error-related potential is detected, a negative penalty physiological feedback reward signal is generated; when the exoskeleton completes the action and no error-related potential is detected, a positive incentive physiological feedback reward signal is generated; when no action is performed, a zero-value physiological feedback reward signal is generated.

[0090] By inputting the generated physiological feedback reward signal into the reward function of the reinforcement learning agent, the agent's evaluation mechanism is updated, guiding the agent to adjust subsequent parameter optimization strategies, reducing the probability of decoding errors, and realizing online self-correction of the system in unlabeled scenarios.

[0091] In some embodiments, the step of inputting physical indicators, including the joint angles of the exoskeleton, into the reinforcement learning agent, and combining EEG signals and physical feedback to backpropagate and optimize the decoding parameters of the EEG decoder, includes: inputting physical indicators, including exoskeleton joint angles, angular velocity, jerk terms, human-computer interaction torque, and motor power consumption, into the reward function of the reinforcement learning agent to calculate physical smoothness rewards and energy consumption rewards; and optimizing the decoding parameters of the EEG decoder through a backpropagation algorithm to ensure that the exoskeleton motion smoothness and energy output stability meet preset standards.

[0092] This embodiment is the core of the physical feedback closed loop of the entire method. The core is to complete the human-computer interaction closed loop missing in the traditional one-way brain control mode. It solves the problems of existing technology ignoring the impact of the exoskeleton's execution state on the user's neural signal feedback, mechanical vibration and human-computer confrontation interference. It achieves deep integration of physiological signals and physical feedback, reverse optimizes decoding parameters, and improves the smoothness and comfort of exoskeleton control.

[0093] In this embodiment, the full-dimensional physical operation indicators of the exoskeleton are directly input into the reward function of the reinforcement learning agent. Based on the physical indicators, physical smoothness rewards and energy consumption rewards are calculated. Combined with the physiological feedback of EEG signals, the decoding parameters of the EEG decoder are optimized through the backpropagation algorithm, so that the exoskeleton's motion smoothness, human-computer interaction friendliness, and energy output stability all meet the preset rehabilitation standards.

[0094] After standardizing and preprocessing the physical indicators such as joint angles, angular velocities, jerk terms, human-computer interaction torque, and instantaneous power consumption change rate of the motor collected by the exoskeleton, they are directly input into the reward function of the reinforcement learning agent.

[0095] Based on the input physical indicators, physical smoothness reward and energy consumption reward are calculated. The smoother the exoskeleton movement, the lower the jerk value, the smaller the fluctuation of the human-machine interaction torque, and the more stable the change of the instantaneous power consumption of the motor, the higher the corresponding reward value. Conversely, the more obvious the movement tremor, the stronger the human-machine confrontation, and the greater the power consumption fluctuation, the lower the corresponding reward value.

[0096] The calculated physical smoothing reward and energy consumption reward, combined with the physiological feedback reward generated in the example, are used to form a composite reward signal. The decoding parameters of the EEG decoder are iteratively optimized through the backpropagation algorithm, adjusting core parameters such as feature weights, classification thresholds, and feature extraction coefficients during the decoding process.

[0097] The smoothness of movement, fluctuation of human-computer interaction torque, and stability of energy output of the optimized exoskeleton are continuously monitored. When all relevant indicators meet the preset rehabilitation standards, the current optimized decoding parameters are locked. If the standards are not met, iterative optimization is continuously carried out until the preset requirements are met.

[0098] In some embodiments, the step of optimizing the decoding parameters corresponding to the multi-dimensional state information based on the real-time performance of each user to generate dynamically evolving control logic includes: using a linear decay mechanism to dynamically adjust the random exploration probability and action noise intensity of the reinforcement learning agent, so that the system can smoothly transition from the extensive exploration stage to the precise adaptation stage; and using a soft update mechanism to slowly synchronize the target network parameters to avoid the exoskeleton from running out of control due to drastic changes in network weights.

[0099] This embodiment is the core of the personalized adaptation of the entire method. The core is to solve the pain points of high cost and heavy burden of manual parameter adjustment in the traditional fixed control mode. At the same time, it ensures the operational stability of the system during the online learning process, avoids exoskeleton loss of control caused by drastic changes in network parameters, and realizes dynamic personalized adaptive control for different users and different rehabilitation stages.

[0100] This embodiment employs a linear decay mechanism to dynamically adjust the random exploration probability and action noise intensity of the reinforcement learning agent, allowing the system to smoothly transition from the extensive exploration phase in the early stage to the precise adaptation phase in the later stage. At the same time, a soft update mechanism is used to slowly synchronize the target network parameters to avoid the exoskeleton from running out of control due to drastic changes in network weights. The decoding parameters are continuously optimized based on the user's real-time performance, generating dynamically evolving personalized control logic for each user.

[0101] By setting an initial random exploration probability for the reinforcement learning agent and a corresponding decay step size, the random exploration probability linearly decreases with running time and training progress during system operation until it reaches a preset minimum exploration threshold. In the early stages of system operation, a higher random exploration probability helps find the optimal decoding parameter range for the user; in the later stages, the exploration probability is reduced to achieve precise adaptation to the user's personalized features. An initial action noise intensity is set for the action vector output by the reinforcement learning agent, along with a corresponding noise decay step size. During system operation, the action noise intensity linearly decreases with running time until it reaches a preset minimum noise threshold, ensuring fine-grained parameter adjustment in the later stages of system operation and avoiding control fluctuations caused by large parameter adjustments.

[0102] During the network update process of the reinforcement learning agent, a soft update mechanism is adopted. Parameters between the policy network, value network, and target network are slowly synchronized according to a preset soft update coefficient. This avoids exoskeleton malfunction due to drastic changes in network weights, ensuring the stability of the system's online learning process. Real-time user performance data is continuously collected, including error-related potential trigger frequency, decoding accuracy, rehabilitation task completion rate, and motion smoothness. Based on this data, decoding parameters are continuously optimized, allowing the control logic to dynamically evolve in sync with the user's neural state, rehabilitation progress, and physical condition, generating personalized control logic for each user that is continuously optimized.

[0103] In some embodiments, the method further includes: introducing time-dependent noise into the action vector output by the reinforcement learning agent to avoid mechanical shaking of the exoskeleton caused by the adjustment of EEG decoder parameters; wherein the noise intensity gradually decreases with training time to ensure the refinement of subsequent parameter adjustment and the stability of the control process.

[0104] This embodiment is a key component for ensuring the stability of system operation. Its core function is to address the issues of exoskeleton mechanical vibration and human-machine interaction caused by parameter mutations during the adjustment of EEG decoder parameters. It aims to balance the exploratory nature of reinforcement learning online learning with the stability of exoskeleton operation, thereby improving the safety and user comfort of the rehabilitation process.

[0105] In this embodiment, time-dependent smooth noise is introduced into the action vector output by the reinforcement learning agent to replace the irregular random white noise, thus avoiding abrupt changes during parameter adjustment. At the same time, a noise intensity attenuation mechanism is set up so that the noise intensity gradually decreases with training time, ensuring the system's early exploration capabilities while ensuring the fine-tuning of parameters and the stability of the control process in the later stages.

[0106] In the policy network of a reinforcement learning agent, time-dependent smooth noise is introduced into the output action vector. This noise has mean-regression properties and will not exhibit irregular and drastic jumps. It can effectively avoid sudden changes in EEG decoder parameters caused by random noise, thereby preventing mechanical shaking of the exoskeleton caused by parameter adjustment.

[0107] An initial noise intensity is set for the introduced smooth noise, along with a corresponding decay step size. During system operation, the noise intensity gradually decays as training time progresses. In the early stages of system operation, a higher noise intensity is used to ensure the exploration range of the agent and help the system quickly find the optimal parameter range. In the later stages of system operation, the noise intensity is gradually reduced to achieve fine-tuning of parameters and ensure a smooth control process.

[0108] During system operation, the mechanical vibration of the exoskeleton and the smoothness of parameter adjustment are continuously monitored. If the mechanical vibration exceeds the preset threshold, the current noise intensity and parameter update step size are automatically reduced to ensure that the exoskeleton operates within a stable and safe range.

[0109] In some embodiments, the basic concept of this invention is to model the parameter optimization process of the EEG decoder as a Markov decision process (MDP) and utilize a reinforcement learning agent to search online for the optimal set of decoding hyperparameters. The system collects data containing EEG features... Physical state of the exoskeleton and current decoding intent Multidimensional state space This provides a comprehensive basis for decision-making. To ensure the smoothness of control, the system introduces time-dependent Ornstein-Uhlenbeck process noise into the action output to avoid mechanical jitter caused by parameter adjustment. In terms of the evaluation mechanism, the system constructs a composite reward function, focusing on introducing a physically smooth reward based on joint jerk (Jerk) and energy consumption terms. And error correction rewards based on physiological feedback This concept enables a higher level of control, moving from "signal decoding" to "performance optimization," giving the system the ability to self-evolve online.

[0110] This approach models the parameter optimization process of the EEG decoder as a Markov Decision Process (MDP), and we model the system as a quadruple. Its core logic is to use a reinforcement learning (RL) agent to adjust the decoder hyperparameters online in order to achieve personalized adaptation.

[0111] Here, S is the state space in the Markov decision process, which is the complete set of all observable valid states of the system, including three core observation dimensions: EEG features, exoskeleton physical state, and decoding intent results. A is the action space in the Markov decision process, which is the complete set of all legal hyperparameter adjustment actions that the reinforcement learning agent can execute, including three dimensions: threshold adjustment, channel weight increment, and feature extraction coefficient correction. T is the state transition function in the Markov decision process, used to describe the probability distribution of the system transitioning from the current state to the next state after the agent performs an action. R is the reward function in the Markov decision process, used to quantify the immediate reward obtained by the agent after performing an action, and is the core evaluation indicator guiding the direction of policy optimization.

[0112] The reinforcement learning agent in this system outputs an action vector at each decision step. It is not directly mapped to the underlying execution torque of the exoskeleton motors, but is defined as an online correction amount for the hyperparameter set inside the EEG decoder. The system adjusts this based on the updated decision threshold. Determine whether a valid brainwave control command has been generated. The logical judgment is as follows: ;in, When the decoder recognizes the user's clear intention to move, it sends the target motion control command to the exoskeleton control system, which corresponds to the preset rehabilitation / assistive action that the exoskeleton needs to perform. It is a standby hold command sent by the decoder to the exoskeleton control system when the decoder does not recognize a valid user movement intention. It is used to control the exoskeleton to maintain the current movement state and avoid accidental triggering of actions.

[0113] When the ambient noise is high Increase the size to prevent accidental touches; when the user's intention is clear and assistance is urgently needed. Reduced to enhance response. The system achieves this through a feature extraction process. Processing current EEG characteristics Calculate the probability distribution of intent classification The specific calculations are as follows: ; Where K is the total number of feature channels involved in EEG intention decoding, corresponding to the number of effective electrode channels in EEG signal acquisition. Softmax is a normalization exponential function used to map the output of the weighted sum of features to a probability distribution in the 0-1 interval with a sum of 1, thereby realizing the confidence quantification of multi-class motion intention.

[0114] State space: the system at time t. Observed state The following common letter identifiers are used, and they are composed of multi-dimensional feature vectors: ; in It is a real-time extracted EEG feature vector, such as the power spectral density (PSD) of a specific frequency band or the energy feature after spatial filtering; This represents the physical feedback from the exoskeleton sensors, where For the angle of the joint, Angular velocity, The physical countermeasure values ​​returned by the torque sensor for human-computer interaction; It represents the initial intent classification result and its confidence distribution output by the decoder at the current moment. For the current category tag, This represents the probability distribution (confidence level) output by the decoder.

[0115] Action space: Action It refers to the dynamic adjustment amount of the decoder's internal hyperparameters: ; in It is an adjustment amount for the classification decision probability threshold, used to balance the sensitivity and accuracy of recognition; It is the incremental allocation of feature channel weights, used to suppress channels with high noise interference and enhance the features of the main graph; These are the coefficient correction values ​​for adaptive feature extraction algorithms (such as spatial filters). Received action Subsequently, the evolution logic of the parameter set follows the state update formula: ;in, and It is the core optimizable hyperparameter set of the EEG decoder at time t and time t+1, which fully includes three major categories of core parameters: classification decision probability threshold Γ, feature channel weight w, and adaptive feature extraction algorithm coefficient ρ. The learning rate is preset. This logic ensures that the decoder can smoothly migrate towards the optimal parameter range based on the current physiological / physical feedback. The policy network employs a stochastic policy function with an exploration threshold and noise to balance the stability and exploratory nature of online learning. ; in, These are the weight parameters of the Actor policy network, used to fit the optimal deterministic hyperparameter adjustment action under the current state, and are the core object of policy network optimization. middle, Indicates a uniform distribution; This represents the lower limit of the action space, i.e., the minimum legal value of the hyperparameter adjustment. This represents the upper limit of the action space, i.e., the maximum legal value of the hyperparameter adjustment; this formula is used to generate random exploration actions within the legal range of the action space. This represents uniformly distributed random numbers generated in real time. This represents the GreedyCoefficient, which decays as the training progresses. When the random number is less than this threshold, the system performs random exploration; otherwise, it executes the recommended action of the Actor network. This represents the deterministic parameter adjustment action of the Actor network output; This involves introducing action space noise to increase the smoothness of actions and the exploration range. To ensure that online learning can escape local optima, the system introduces an Ornstein-Uhlenbeck process, as detailed below: ; It is the time derivative, corresponding to the time interval of a single decision step in the system. In the integral term, it is the integral time derivative, used for kinematics and reward value calculation in the continuous time domain. It is the differential increment of the Wiener process (standard Brownian motion), used to generate the random fluctuation term in the Ornstein-Uhlenbeck process, characterizing the random noise caused by user neural activity and environmental disturbances.

[0116] State transition function: The state transition function defines the probability density of the system entering the next state after performing parameter adjustment actions. ; in This represents a complex human-machine coupled environment model; This represents the random fluctuations in the user's neural activity. The function describes how parameter adjustments alter the decoding accuracy of the next step, thereby influencing the overall state at the next moment through the physical motion feedback of the exoskeleton.

[0117] Reward function: Reward function Used to guide the parameters to converge in the optimal direction: ; in It is a physiological feedback reward that uses real-time monitoring of error-related potentials (ErrPs) in the brain. If an ErrP is detected, it indicates a decoding error, and a penalty is imposed. Conversely, positive rewards are given, in the following forms: ; It is a fixed penalty coefficient corresponding to decoding errors. When the system detects the error-related potential ErrP, it applies this negative penalty to the agent to suppress the error parameter adjustment strategy. It correctly decodes the corresponding fixed reward coefficient. When the system performs parameter adjustment actions and does not detect the error-related potential ErrP, it applies a positive reward of that value to the agent to reinforce the correct parameter adjustment strategy. It is a physical smoothness reward, which evaluates the smoothness of movement through kinematic data. The smoother the movement and the smaller the interaction force, the higher the reward value. To achieve higher accuracy, a composite index of fourth derivative (Jerk) and energy consumption is introduced: ; It is the third derivative of joint position with respect to time, describing the tremor. It represents the deviation between the real-time interactive torque and the moving average torque, describing sudden physical confrontation. It is the rate of change of instantaneous power consumption of the motor, describing the smoothness of energy output. is the weighting coefficient; N is the length of the sliding time window used to calculate motion smoothness, that is, the total number of consecutive decision steps involved in smoothness evaluation. The larger the window, the wider the time span of smoothness evaluation. It is the time step index within the sliding time window, used to traverse all consecutive moments within the window and complete the summation calculation of smoothness-related indicators. It is the L2 norm (Euclidean norm), used to quantify the magnitude of a vector. Here, it is used to calculate the absolute magnitude of joint jerk and torque deviation. It is the moving average of the human-computer interaction torque, used to characterize the stable baseline of the human-computer interaction torque. By subtracting it from the real-time interaction torque, it quantifies sudden physical confrontation situations. It is a reward discount factor in reinforcement learning, used to weigh the importance of future rewards against current immediate rewards. Its value ranges from 0 to 1, and the larger the value, the more the agent focuses on long-term gains.

[0118] This is a task efficiency reward, given based on the progress of completing the preset rehabilitation trajectory. It consists of the remaining distance to the rehabilitation target location and the integral term of the trajectory tracking error, and its specific form is as follows: ; in, Indicates the remaining distance to the target rehabilitation location. The task progress weighting coefficient adjusts the importance of "speed" in the rewards. It is the trajectory tracking penalty coefficient; This represents the total Euclidean distance from the starting point to the target point, pre-set before the rehabilitation task begins; This represents the real-time remaining distance between the current position of the exoskeleton's end or joint and the rehabilitation target at time t. This represents the total time elapsed from the start of the rehabilitation task to the current time t. These are standard joint angles, a preset reference angle sequence for rehabilitation programs; It is the actual joint angle.

[0119] Training initialization: Perform the following initialization operations when the system starts up or a new patient is connected: Network initialization: Randomly initialize the Actor network of the agent. Critic Network Target network weights Initialize the learning rate, including: the Actor network learning rate. Critic Network Learning Rate Discount factor Soft update coefficient Training batch size .

[0120] Initialize the following exploration and noise-related parameters, setting the initial greedy coefficient to [value]. The greed coefficient decay step size is OU noise parameters include: noise regression rate. Long-term noise average Noise fluctuation intensity Initial motion noise intensity Noise attenuation step size .

[0121] The initial state of the EEG decoder includes: the initial classification decision probability threshold. Initial feature channel weight allocation vector Initial feature extraction algorithm coefficients (e.g., initial CSP filter coefficients); decoding parameter update step size and learning rate (control Acting on (Intensity).

[0122] Reward function weighting factor: Setting Corresponding to physiological feedback rewards Physical smoothing reward and task efficiency rewards Weighting coefficients; These correspond to the proportional coefficients of the Jerk term, torque deviation term, and power consumption term in the physical smoothing reward, respectively. These correspond to the ratio coefficients of the progress item and the accuracy penalty item in the task efficiency reward. Specific factor information is shown in the table below: Network Update: Experience Replay Pool Interaction trajectory tuples stored in ,when Reaching the preset batch processing threshold At this time, the system initiates an asynchronous background update process. This process continuously optimizes the decoding parameter adjustment strategy by minimizing value loss and maximizing the policy gradient.

[0123] The system retrieves data from the experience replay pool. The size of the random sampling is Sample batches are used to break down correlations between data. Each sample contains the current state. , the adjustment actions performed The compound rewards obtained And the next state after evolution .

[0124] The system utilizes a target Actor network. and target Critic network Calculate the estimated value of the next state and complete the Critic network update: ; in The discount factor set for the initialization phase.

[0125] By minimizing the mean square error loss function Update Critic network weights : ;in, It is the Temporal Difference Target (TDTarget), which is calculated by combining the immediate composite reward obtained at the current moment with the target network's predicted value for the next state. It serves as the true target value for the Critic network and is used to update the Critic network weights. It is the mean squared error loss function of the Critic value network, used to quantify the deviation between the value prediction result of the Critic network and the time-series difference target value. The weight optimization of the Critic network is achieved by minimizing this loss function.

[0126] The system adjusts the weights using the Policy Gradient algorithm. This allows the output action to achieve a higher Critic score: ; It is the gradient operator, used to calculate the partial derivative of the objective function with respect to the corresponding network parameters. It is the core operator for backpropagation and parameter update in neural networks. It is the objective function of policy optimization, used to quantify the overall performance of the Actor policy network. The core objective of policy optimization is to maximize this objective function so that the actions output by the agent can obtain higher long-term cumulative rewards.

[0127] To maintain stability during online training, the system employs a soft update mechanism to slowly synchronize the target network parameters. To avoid the exoskeleton from losing control due to drastic changes in network weights: ;in The soft update coefficient set for the initialization phase.

[0128] To enable the system to smoothly transition from the early "extensive exploration" phase to the later "precise adaptation" phase, a two-factor control logic that decays linearly with time step t is introduced.

[0129] The linear decay control logic includes: the greedy coefficient decay is controlled by adjusting the random exploration probability. The time decreases gradually as the episode progresses: ; in The initial greed coefficient, To decay step size, It is the minimum lower bound of the greed coefficient, used to limit the minimum probability of random exploration by the system, and to prevent the greed coefficient from decaying to 0 in the later stages of training, causing the system to completely lose its exploration ability.

[0130] Action noise intensity attenuation is achieved by controlling the influence range of OU noise. This ensures fine-grained parameter adjustments in the later stages. ; in The initial motion noise intensity, This is the noise attenuation step size. It is the minimum lower limit of motion noise intensity, used to limit the minimum impact of OU noise, and to prevent the noise intensity from decaying to 0 in the later stages of training, which would prevent the system from escaping the local optimum.

[0131] During the sampling process, the system extracts feature vectors in real time from PSD (Power Spectral Density) and CSP (Common Spatial Pattern) spatial filtering to construct the state space. Simultaneously, the system needs to monitor ErrP (Error-Related Potential), an intrinsic error-correcting signal, and use it as a physiological feedback reward. The triggering basis is determined by calculating the probability distribution output by the decoder. With its confidence level, the system completes the assessment of the current neural state. Quantitative observation.

[0132] In terms of physical motion monitoring, the system acquires the angles of each joint of the exoskeleton through joint angle / angular velocity sensors (such as encoders or IMUs). With angular velocity To achieve refined motion evaluation, the system performs higher-order derivatives on the position signal to calculate the Jerk (jerk) term describing the flutter. These physical parameters constitute the state vector. These metrics are used to describe limb dynamics in real-time human-machine coupled environments. They are directly input into the reward function. This item is used to evaluate the smoothness of the control strategy.

[0133] To address the issues of human-computer interaction mechanics and energy feedback, the system deploys a TorqueSensor to collect the torques involved in human-computer interaction. The system quantifies sudden physical confrontation situations by calculating the deviation between the real-time interactive torque and the moving average torque. In addition, the system also monitors the instantaneous rate of change of the motor's power consumption. This is used as a key indicator for evaluating the stability of energy output. During mission execution, the system continuously tracks the remaining distance to the target. By combining the trajectory tracking error calculation to TaskEfficiencyReward, the progress and accuracy of rehabilitation training can be ensured.

[0134] Please see Figure 4 As shown, Figure 4This is a schematic diagram of the structure of the brain-computer interface exoskeleton multidimensional feedback adaptive control system 200 provided in this application embodiment. The brain-computer interface exoskeleton multidimensional feedback adaptive control system 200 is used to execute the steps of the brain-computer interface exoskeleton multidimensional feedback adaptive control method shown in the above embodiments. The brain-computer interface exoskeleton multidimensional feedback adaptive control system 200 can be a single server or a server cluster, or it can be a terminal, such as a handheld terminal, a laptop computer, a wearable device, or a robot.

[0135] like Figure 4 As shown, the brain-computer interface exoskeleton multidimensional feedback adaptive control system 200 includes: Information acquisition unit 201 is used to acquire multi-dimensional state information including EEG characteristics, exoskeleton physical state and current decoding intent, and construct a multi-dimensional state space based on the multi-dimensional state information; The parameter correction unit 202 is used to adjust the hyperparameters of the EEG decoder using a reinforcement learning agent, so as to model the parameter optimization process of the EEG decoder as a decision-making process; and to correct the hyperparameter set inside the EEG decoder by outputting the action vector corresponding to the multi-dimensional state information through the reinforcement learning agent. The parameter optimization unit 203 is used to detect the error-related potentials automatically triggered by the brain due to the inconsistency between the exoskeleton's movements and the user's intentions based on the multidimensional state information, and use them as internal reward signals to correct the multidimensional state information; input physical indicators including the joint angles of the exoskeleton into the reinforcement learning agent, and combine the EEG signals and physical feedback to reverse optimize the decoding parameters of the EEG decoder. The control completion unit 204 is used to optimize the decoding parameters corresponding to the multi-dimensional state information based on the real-time performance of each user, generate dynamically evolving control logic, and realize adaptive personalized control.

[0136] In some embodiments, the acquisition of multidimensional state information including EEG features, exoskeleton physical state, and current decoding intent includes: extracting power spectral density features and feature vectors based on common space pattern spatial filtering of EEG signals in real time using an EEG acquisition device as EEG features; acquiring the angles, angular velocities, human-computer interaction torques, instantaneous power consumption change rates of motors, and jerk terms of each joint of the exoskeleton using joint angle sensors, angular velocity sensors, torque sensors, and motor power consumption monitoring modules as the exoskeleton physical state; and calculating the intent classification probability distribution and its confidence level output by the EEG decoder as the current decoding intent.

[0137] In some embodiments, constructing a multidimensional state space based on the multidimensional state information includes: taking EEG features, exoskeleton physical state, and current decoding intent as three components of the multidimensional state space, and combining them to form a multidimensional feature vector containing a time series; wherein the EEG feature component is used to characterize the user's neural signal state, the exoskeleton physical state component is used to characterize the human-machine coupling dynamics, and the current decoding intent component is used to characterize the reliability assessment of the current control command.

[0138] In some embodiments, the step of using a reinforcement learning agent to adjust the hyperparameters of the EEG decoder to model the parameter optimization process of the EEG decoder as a decision process includes: modeling the parameter optimization process of the EEG decoder as a Markov decision process, defining a quadruple model including a state space, an action space, transition probabilities, and a reward function, and selecting an adjustment strategy for the hyperparameters of the EEG decoder based on the input of the multidimensional state space by the reinforcement learning agent, with the reward signal from the environmental feedback as the optimization target.

[0139] In some embodiments, the step of correcting the hyperparameter set inside the EEG decoder by outputting action vectors corresponding to multidimensional state information through a reinforcement learning agent includes: using the action vectors output by the reinforcement learning agent as the online correction amount for the hyperparameter set inside the EEG decoder, and determining whether to generate a valid EEG control command based on the corrected decision threshold; wherein the decision threshold is dynamically adjusted according to the level of environmental noise and the clarity of the user's intent, increasing the threshold when the environmental noise is high to prevent accidental touches, and decreasing the threshold when the user's intent is clear to enhance the response.

[0140] In some embodiments, the step of detecting error-related potentials automatically triggered by the brain due to discrepancies between exoskeleton movements and user intentions based on the multidimensional state information, and using these as internal reward signals to correct the multidimensional state information, includes: analyzing the characteristic waveforms of error-related potentials in the electroencephalogram (EEG) signal, calculating the deviation between the intention classification probability distribution output by the decoder and the actual action feedback, and determining that an error-related potential is triggered when the deviation exceeds a preset threshold, generating a physiological feedback reward signal to update the evaluation mechanism of the reinforcement learning agent.

[0141] In some embodiments, the step of inputting physical indicators, including the joint angles of the exoskeleton, into the reinforcement learning agent, and combining EEG signals and physical feedback to backpropagate and optimize the decoding parameters of the EEG decoder, includes: inputting physical indicators, including exoskeleton joint angles, angular velocity, jerk terms, human-computer interaction torque, and motor power consumption, into the reward function of the reinforcement learning agent to calculate physical smoothness rewards and energy consumption rewards; and optimizing the decoding parameters of the EEG decoder through a backpropagation algorithm to ensure that the exoskeleton motion smoothness and energy output stability meet preset standards.

[0142] In some embodiments, the step of optimizing the decoding parameters corresponding to the multi-dimensional state information based on the real-time performance of each user to generate dynamically evolving control logic includes: using a linear decay mechanism to dynamically adjust the random exploration probability and action noise intensity of the reinforcement learning agent, so that the system can smoothly transition from the extensive exploration stage to the precise adaptation stage; and using a soft update mechanism to slowly synchronize the target network parameters to avoid the exoskeleton from running out of control due to drastic changes in network weights.

[0143] In some embodiments, the method further includes: introducing time-dependent noise into the action vector output by the reinforcement learning agent to avoid mechanical shaking of the exoskeleton caused by the adjustment of EEG decoder parameters; wherein the noise intensity gradually decreases with training time to ensure the refinement of subsequent parameter adjustment and the stability of the control process.

[0144] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the brain-computer interface exoskeleton multidimensional feedback adaptive control system and its modules described above can be found in the corresponding contents of the various embodiments of the brain-computer interface exoskeleton multidimensional feedback adaptive control method, and will not be repeated here.

[0145] The aforementioned brain-computer interface exoskeleton multidimensional feedback adaptive control method can be implemented as a computer program, which can be used in, for example... Figure 4 It runs on the device shown.

[0146] Please see Figure 5 , Figure 5 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application. The computer device includes a processor, a memory, and a network interface connected via a device bus, wherein the memory may include a storage medium and internal memory.

[0147] The storage medium can store the operating device and the computer program. The computer program includes program instructions that, when executed, cause the processor to perform any multidimensional feedback adaptive control method for the brain-computer interface exoskeleton.

[0148] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0149] Internal memory provides an environment for the execution of computer programs in non-volatile storage media. When the computer program is executed by the processor, it enables the processor to execute any multidimensional feedback adaptive control method for brain-computer interface exoskeletons.

[0150] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 5The structure shown is merely a block diagram of a portion of the structure related to the solution of this application, and does not constitute a limitation on the terminal on which the solution of this application is applied. Specific computer devices may include, but are not limited to, those shown in the diagram. Figure 5 The diagram shows more or fewer components, or combinations of certain components, or different component arrangements.

[0151] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0152] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: Collect multidimensional state information including EEG features, exoskeleton physical state, and current decoding intent, and construct a multidimensional state space based on the multidimensional state information; The hyperparameters of the EEG decoder are adjusted by using a reinforcement learning agent, so as to model the parameter optimization process of the EEG decoder as a decision-making process; the action vectors corresponding to the multi-dimensional state information output by the reinforcement learning agent are used to correct the hyperparameter set of the EEG decoder. The multidimensional state information is used to detect the error-related potentials automatically triggered by the brain due to the inconsistency between the exoskeleton's movements and the user's intentions. These potentials are then used as internal reward signals to correct the multidimensional state information. Physical indicators, including the joint angles of the exoskeleton, are input into the reinforcement learning agent. Combined with EEG signals and physical feedback, the decoding parameters of the EEG decoder are optimized in reverse. Based on each user's real-time performance, the decoding parameters corresponding to the multi-dimensional state information are optimized to generate dynamically evolving control logic, thereby achieving adaptive personalized control.

[0153] In some embodiments, the acquisition of multidimensional state information including EEG features, exoskeleton physical state, and current decoding intent includes: extracting power spectral density features and feature vectors based on common space pattern spatial filtering of EEG signals in real time using an EEG acquisition device as EEG features; acquiring the angles, angular velocities, human-computer interaction torques, instantaneous power consumption change rates of motors, and jerk terms of each joint of the exoskeleton using joint angle sensors, angular velocity sensors, torque sensors, and motor power consumption monitoring modules as the exoskeleton physical state; and calculating the intent classification probability distribution and its confidence level output by the EEG decoder as the current decoding intent.

[0154] In some embodiments, constructing a multidimensional state space based on the multidimensional state information includes: taking EEG features, exoskeleton physical state, and current decoding intent as three components of the multidimensional state space, and combining them to form a multidimensional feature vector containing a time series; wherein the EEG feature component is used to characterize the user's neural signal state, the exoskeleton physical state component is used to characterize the human-machine coupling dynamics, and the current decoding intent component is used to characterize the reliability assessment of the current control command.

[0155] In some embodiments, the step of using a reinforcement learning agent to adjust the hyperparameters of the EEG decoder to model the parameter optimization process of the EEG decoder as a decision process includes: modeling the parameter optimization process of the EEG decoder as a Markov decision process, defining a quadruple model including a state space, an action space, transition probabilities, and a reward function, and selecting an adjustment strategy for the hyperparameters of the EEG decoder based on the input of the multidimensional state space by the reinforcement learning agent, with the reward signal from the environmental feedback as the optimization target.

[0156] In some embodiments, the step of correcting the hyperparameter set inside the EEG decoder by outputting action vectors corresponding to multidimensional state information through a reinforcement learning agent includes: using the action vectors output by the reinforcement learning agent as the online correction amount for the hyperparameter set inside the EEG decoder, and determining whether to generate a valid EEG control command based on the corrected decision threshold; wherein the decision threshold is dynamically adjusted according to the level of environmental noise and the clarity of the user's intent, increasing the threshold when the environmental noise is high to prevent accidental touches, and decreasing the threshold when the user's intent is clear to enhance the response.

[0157] In some embodiments, the step of detecting error-related potentials automatically triggered by the brain due to discrepancies between exoskeleton movements and user intentions based on the multidimensional state information, and using these as internal reward signals to correct the multidimensional state information, includes: analyzing the characteristic waveforms of error-related potentials in the electroencephalogram (EEG) signal, calculating the deviation between the intention classification probability distribution output by the decoder and the actual action feedback, and determining that an error-related potential is triggered when the deviation exceeds a preset threshold, generating a physiological feedback reward signal to update the evaluation mechanism of the reinforcement learning agent.

[0158] In some embodiments, the step of inputting physical indicators, including the joint angles of the exoskeleton, into the reinforcement learning agent, and combining EEG signals and physical feedback to backpropagate and optimize the decoding parameters of the EEG decoder, includes: inputting physical indicators, including exoskeleton joint angles, angular velocity, jerk terms, human-computer interaction torque, and motor power consumption, into the reward function of the reinforcement learning agent to calculate physical smoothness rewards and energy consumption rewards; and optimizing the decoding parameters of the EEG decoder through a backpropagation algorithm to ensure that the exoskeleton motion smoothness and energy output stability meet preset standards.

[0159] In some embodiments, the step of optimizing the decoding parameters corresponding to the multi-dimensional state information based on the real-time performance of each user to generate dynamically evolving control logic includes: using a linear decay mechanism to dynamically adjust the random exploration probability and action noise intensity of the reinforcement learning agent, so that the system can smoothly transition from the extensive exploration stage to the precise adaptation stage; and using a soft update mechanism to slowly synchronize the target network parameters to avoid the exoskeleton from running out of control due to drastic changes in network weights.

[0160] In some embodiments, the method further includes: introducing time-dependent noise into the action vector output by the reinforcement learning agent to avoid mechanical shaking of the exoskeleton caused by the adjustment of EEG decoder parameters; wherein the noise intensity gradually decreases with training time to ensure the refinement of subsequent parameter adjustment and the stability of the control process.

[0161] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the multidimensional feedback adaptive control method for brain-computer interface exoskeleton as provided in any embodiment of this application.

[0162] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0163] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multidimensional feedback adaptive control method for a brain-computer interface exoskeleton, characterized in that, include: Collect multidimensional state information including EEG features, exoskeleton physical state, and current decoding intent, and construct a multidimensional state space based on the multidimensional state information; By using reinforcement learning agents to adjust the hyperparameters of the EEG decoder, the parameter optimization process of the EEG decoder can be modeled as a decision-making process. The action vectors corresponding to the multi-dimensional state information output by the reinforcement learning agent are used to correct the hyperparameter set inside the EEG decoder. The multidimensional state information is used to detect the error-related potentials automatically triggered by the brain due to the inconsistency between the exoskeleton's movements and the user's intentions. These potentials are then used as internal reward signals to correct the multidimensional state information. Physical indicators, including the joint angles of the exoskeleton, are input into the reinforcement learning agent. Combined with EEG signals and physical feedback, the decoding parameters of the EEG decoder are optimized in reverse. Based on each user's real-time performance, the decoding parameters corresponding to the multi-dimensional state information are optimized to generate dynamically evolving control logic, thereby achieving adaptive personalized control.

2. The method according to claim 1, characterized in that, The data acquisition includes multidimensional state information such as EEG characteristics, exoskeleton physical state, and current decoding intent, including: The power spectral density features and feature vectors based on common spatial pattern spatial filtering of EEG signals are extracted in real time using EEG acquisition equipment and used as EEG features. By using joint angle sensors, angular velocity sensors, torque sensors, and motor power consumption monitoring modules, the angles, angular velocities, human-machine interaction torques, instantaneous power consumption change rates of motors, and jerk terms of each joint of the exoskeleton are obtained as the physical state of the exoskeleton. The current decoded intent is determined by calculating the probability distribution of intent classification and its confidence level output by the EEG decoder.

3. The method according to claim 1, characterized in that, The construction of the multidimensional state space based on the multidimensional state information includes: EEG features, exoskeleton physical state, and current decoding intent are each taken as three components of a multidimensional state space and combined to form a multidimensional feature vector containing time series. Among them, the EEG feature component is used to characterize the user's neural signal state, the exoskeleton physical state component is used to characterize the human-machine coupling dynamics, and the current decoded intent component is used to characterize the reliability assessment of the current control command.

4. The method according to claim 1, characterized in that, The method of using reinforcement learning agents to adjust the hyperparameters of the EEG decoder, thereby modeling the parameter optimization process of the EEG decoder as a decision-making process, includes: The parameter optimization process of the EEG decoder is modeled as a Markov decision process. A four-tuple model is defined, which includes a state space, an action space, transition probabilities, and a reward function. The reinforcement learning agent selects an adjustment strategy for the hyperparameters of the EEG decoder based on the input of the multi-dimensional state space, and uses the reward signal from the environmental feedback as the optimization target.

5. The method according to claim 1, characterized in that, The method of using a reinforcement learning agent to output action vectors corresponding to multi-dimensional state information to correct the hyperparameter set within the EEG decoder includes: The action vector output by the reinforcement learning agent is used as the online correction amount of the hyperparameter set inside the EEG decoder. The decision threshold after correction is used to determine whether to generate an effective EEG control command. The decision threshold is dynamically adjusted based on the level of environmental noise and the clarity of the user's intent. When the environmental noise is high, the threshold is increased to prevent accidental touches, and when the user's intent is clear, the threshold is decreased to enhance the response.

6. The method according to claim 1, characterized in that, The step of detecting error-related potentials automatically triggered by the brain due to discrepancies between exoskeleton movements and user intentions based on the multidimensional state information, and using these potentials as internal reward signals to correct the multidimensional state information, includes: By analyzing the characteristic waveforms of error-related potentials in EEG signals, the deviation between the intention classification probability distribution output by the decoder and the actual action feedback is calculated. When the deviation exceeds a preset threshold, an error-related potential is triggered, and a physiological feedback reward signal is generated to update the evaluation mechanism of the reinforcement learning agent.

7. The method according to claim 1, characterized in that, The process of inputting physical parameters, including the joint angles of the exoskeleton, into the reinforcement learning agent, and combining EEG signals and physical feedback to inversely optimize the decoding parameters of the EEG decoder, includes: Physical indicators, including exoskeleton joint angles, angular velocities, jerk factors, human-computer interaction torque, and motor power consumption, are input into the reward function of the reinforcement learning agent to calculate physical smoothing rewards and energy consumption rewards. The decoding parameters of the EEG decoder are optimized by using the backpropagation algorithm, so that the smoothness of exoskeleton movement and the stability of energy output meet the preset standards.

8. The method according to claim 1, characterized in that, The process of optimizing the decoding parameters corresponding to the multi-dimensional state information based on each user's real-time performance to generate dynamically evolving control logic includes: A linear decay mechanism is used to dynamically adjust the random exploration probability and action noise intensity of the reinforcement learning agent, so that the system can smoothly transition from the extensive exploration stage to the precise adaptation stage. The target network parameters are slowly synchronized through a soft update mechanism to avoid the exoskeleton from running out of control due to drastic changes in network weights.

9. The method according to claim 1, characterized in that, The method further includes: Temporally relevant noise is introduced into the action vectors output by the reinforcement learning agent to avoid mechanical shaking of the exoskeleton caused by EEG decoder parameter adjustment; The noise intensity gradually decreases over time, ensuring finer parameter adjustments and a smoother control process in the later stages.

10. A multi-dimensional feedback adaptive control system for a brain-computer interface exoskeleton, characterized in that, The method applied to any one of claims 1-9 includes: The information acquisition unit is used to acquire multi-dimensional state information including EEG characteristics, exoskeleton physical state and current decoding intent, and to construct a multi-dimensional state space based on the multi-dimensional state information. The parameter correction unit is used to adjust the hyperparameters of the EEG decoder using a reinforcement learning agent, so as to model the parameter optimization process of the EEG decoder as a decision-making process; and to correct the hyperparameter set inside the EEG decoder by outputting action vectors corresponding to multi-dimensional state information through the reinforcement learning agent. The parameter optimization unit is used to detect the error-related potentials automatically triggered by the brain due to the inconsistency between the exoskeleton's movements and the user's intentions based on the multidimensional state information, and use them as internal reward signals to correct the multidimensional state information; it inputs physical indicators, including the joint angles of the exoskeleton, into the reinforcement learning agent, and combines EEG signals and physical feedback to back-optimize the decoding parameters of the EEG decoder. The control completion unit is used to optimize the decoding parameters corresponding to the multi-dimensional state information based on the real-time performance of each user, generate dynamically evolving control logic, and realize adaptive personalized control.

Images