AI-based co-driving exoskeleton shared control methods, systems, devices, and storage media
By using an AI-driven exoskeleton shared control method, combined with local trajectory correction and hybrid long- and short-term motion optimization, the problem of exoskeleton walking in complex three-dimensional environments has been solved, achieving stability and flexibility in omnidirectional walking.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TONGJI UNIV
- Filing Date
- 2024-08-19
- Publication Date
- 2026-07-17
AI Technical Summary
Existing exoskeleton planning and control technologies are mainly used in walking scenarios in barrier-free spaces, and it is difficult to achieve omnidirectional walking in the complex three-dimensional environment full of various obstacles in daily life.
An AI-based co-driving exoskeleton shared control method is adopted. By acquiring the wearer's real-time pose data and switching between local trajectory correction mode and hybrid long-term and short-term motion optimization mode, the path is corrected and optimized using an AI co-driving strategy network model, enabling the exoskeleton to walk omnidirectionally in a complex three-dimensional environment.
When encountering sudden obstacles, the exoskeleton can combine pose data and environmental image information to correct its path through AI, enabling omnidirectional walking and improving mobility and stability in complex environments.
Smart Images
Figure CN119098934B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of exoskeleton technology, and in particular to a shared control method, system, device and storage medium for an exoskeleton based on AI co-driving. Background Technology
[0002] An exoskeleton is a wearable mechanical device that provides users with extra energy for limb movement, improving their walking, lifting, and other activities, and helping to enhance their strength, stability, and motor skills. With advancements in sensors, actuators, and artificial intelligence, the mobility and intelligence of exoskeleton robots have been further improved. As an assistive robot, it can help wearers complete rehabilitation training tasks, assist with daily walking, and provide home walking assistance, alleviating the pressure of caregiver shortages to some extent, reducing high labor costs, and further improving the quality of life for wearers.
[0003] Currently, there is a large number of manual laborers and patients with lower limb motor dysfunction worldwide, creating a huge market and urgent demand for exoskeletons. However, existing exoskeleton planning and control technologies are mainly applied to walking scenarios in barrier-free spaces, and the movement paths are usually limited to fixed directions. In everyday life, however, walking scenarios are often filled with various obstacles, such as furniture and other objects, which pose challenges to the mobility of exoskeletons.
[0004] Therefore, how to enable exoskeletons to achieve omnidirectional walking in complex three-dimensional environments is a technical problem that urgently needs to be solved in this field. Summary of the Invention
[0005] In order to overcome the defects of the above-mentioned related technologies, the present invention provides an exoskeleton sharing control method, system, device and storage medium based on AI co-driving, which realizes omnidirectional walking of exoskeleton in a complex three-dimensional environment through AI co-driving exoskeleton motion optimization and control switching strategy.
[0006] According to one aspect of the present invention, an AI-based co-driving exoskeleton sharing control method is provided, comprising:
[0007] Acquire real-time pose data of exoskeleton wearers;
[0008] Based on the pose data, the local trajectory correction mode and the hybrid long-term and short-term motion optimization mode are switched in real time. The local trajectory correction mode is completed based on the AI co-driving strategy network model.
[0009] Multi-task tracking of the direction and trajectory of the three-dimensional movement of the exoskeleton is performed to achieve motion tracking control of the exoskeleton.
[0010] In some embodiments of this application, the switching between a local trajectory correction mode and a hybrid long-term and short-term motion optimization mode based on the pose data in real time includes:
[0011] Preset wearer posture threshold;
[0012] Compare the pose threshold with the pose data;
[0013] If the pose data is greater than the pose threshold, switch to local trajectory correction mode; if the pose data is less than the pose threshold, switch to hybrid long-term and short-term motion optimization mode.
[0014] In some embodiments of this application, the AI co-driving policy network model includes:
[0015] Obtain image information of the target point;
[0016] Feature extraction is performed on the image information of the target point, and the coordinates and distance of the target point are calculated;
[0017] Input the pose data, target point image feature data, and target point coordinates and distance;
[0018] Based on the input, the increment of the predicted pose data is calculated and output.
[0019] In some embodiments of this application, the hybrid long-short-term motion optimization includes:
[0020] Long-term planning is used to output the optimal global path; the output of the long-term planning is calculated based on an SNN network model; the SNN network model includes an encoding layer and a decoding layer, the encoding layer includes multiple convolutional layers and at least one pooling layer, and the decoding layer includes multiple convolutional layers and at least one upsampling layer;
[0021] Short-term planning is used to output the foot sequence and whole-body motion trajectory of the exoskeleton. The short-term planning includes: pre-defined foot placement information, including foot size, foot constraints, and foot state definition; outputting a set of foot sequences based on foot transfer strategies and iterative updates of left and right feet; and outputting the whole-body motion trajectory based on forward and inverse kinematics analysis.
[0022] Based on the long-term and short-term plans, the three-dimensional motion trajectory of the exoskeleton is generated.
[0023] In some embodiments of this application, the loss function of the SNN network model is expressed as the following formula:
[0024]
[0025] Where a, b, and c represent hyperparameters, P represents the predicted path, T represents the true path, L(·) represents the path length, S(·) represents the similarity between the true path and the predicted path, i represents the i-th row of P and T, and j represents the j-th column of P and T.
[0026] In some embodiments of this application, the multi-task tracking of the direction of three-dimensional movement and walking trajectory of the exoskeleton includes:
[0027] Establish a mapping transformation relationship from joint space to task space;
[0028] Based on the aforementioned mapping transformation relationship, the task space Jacobian is calculated;
[0029] Calculate the augmented Jacobian matrix based on the task space Jacobian;
[0030] Based on the augmented Jacobian matrix, the null space projection operator is calculated;
[0031] Based on the null space projection operator, calculate the decoupled Jacobian matrix and the task speed of hierarchical decoupling;
[0032] Based on the decoupled Jacobian matrix and the hierarchical decoupled task speed, multi-task tracking is performed by the controller.
[0033] According to another aspect of this application, an AI-based co-driving exoskeleton sharing control system is also provided, comprising:
[0034] The pose data processing module is used to acquire real-time pose data of the exoskeleton wearer; and to switch between the local trajectory correction module and the hybrid long-term and short-term motion optimization module in real time based on the pose data, wherein the local trajectory correction module includes an AI co-driving strategy network model.
[0035] The local trajectory correction module is used to output the increment of the predicted pose data to correct the local motion trajectory.
[0036] The hybrid long-term and short-term motion optimization module is used to output the three-dimensional motion trajectory of the exoskeleton;
[0037] The multi-task controller module is used to perform multi-task tracking of the direction and walking trajectory of the exoskeleton's three-dimensional movement, thereby realizing the walking tracking control of the exoskeleton.
[0038] In some embodiments of this application, the pose data processing module includes:
[0039] IMU sensor: used for pose data acquisition;
[0040] IMU acquisition program: used for pose data processing and, based on the pose data, switching between the local trajectory correction module and the hybrid long-term and short-term motion optimization module in real time.
[0041] According to another aspect of this application, an AI-based co-driving exoskeleton sharing control device is also provided, comprising:
[0042] An exoskeleton, the exoskeleton including a DC brushless motor;
[0043] The shared control system as described above;
[0044] The shared control system drives the exoskeleton to move by controlling the brushless DC motor.
[0045] According to another aspect of this application, a computer-readable storage medium is also provided, on which computer instructions are stored, which, when executed by a processor, implement the exoskeleton sharing control method based on AI co-driving as described above.
[0046] Compared with the prior art, the advantages of this invention are:
[0047] The AI-based co-driving exoskeleton shared control method, system, device, and storage medium provided by this invention can, when encountering sudden obstacles during exoskeleton movement, combine pose data and environmental image information to obtain the optimal movement path through AI correction and prediction, thereby helping the exoskeleton achieve omnidirectional walking in complex three-dimensional environments. Attached Figure Description
[0048] The above and other features and advantages of the present invention will become more apparent from a detailed description of exemplary embodiments thereof with reference to the accompanying drawings.
[0049] Figure 1 A flowchart illustrating the AI-based co-driving exoskeleton sharing control method of the present invention is shown.
[0050] Figure 2 A block diagram of the AI-based co-driving exoskeleton sharing control system of the present invention is shown.
[0051] Figure 3 A block diagram of the AI-based co-driving exoskeleton shared control device of the present invention is shown.
[0052] Figure 4 A schematic diagram of the operation of the AI-based co-driving exoskeleton shared control device of the present invention is shown.
[0053] Figure 5 A schematic diagram of the operation of the local trajectory correction module of the present invention is shown.
[0054] Figure 6A schematic diagram of the operation of the hybrid long-short-term motion optimization module of the present invention is shown.
[0055] Figure 7 A schematic diagram of the multitasking controller module of the present invention is shown.
[0056] Figure 8 A schematic diagram of the installation location of the IMU device of the present invention is shown. Detailed Implementation
[0057] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0058] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0059] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined. Therefore, the actual execution order may change depending on the specific circumstances.
[0060] Figure 1 A flowchart illustrating the AI-based co-driving exoskeleton sharing control method of the present invention is shown. The AI-based co-driving exoskeleton sharing control method provided in this application includes the following steps S110-S130:
[0061] Step S110: Obtain real-time pose data of the exoskeleton wearer.
[0062] Specifically, hardware devices such as IMU sensors can be used to monitor and collect the posture data of exoskeleton wearers.
[0063] Step S120: Based on pose data, switch between local trajectory correction mode and hybrid long-term and short-term motion optimization mode in real time. The local trajectory correction mode is completed based on the AI co-driving strategy network model.
[0064] Specifically, an IMU acquisition program can be used to process pose data and, based on the pose data, switch between a local trajectory correction mode and a hybrid long-term and short-term motion optimization mode in real time. The local trajectory correction mode is completed based on an AI co-driving strategy network model. The AI co-driving strategy network model includes: acquiring target point image information; extracting features from the target point image information and calculating the target point coordinates and distance; inputting pose data, target point image feature data, and target point coordinates and distance; and calculating and outputting the increment of the predicted pose data based on the input.
[0065] In some embodiments, a wearer pose threshold is preset in the IMU acquisition program, and then the pose threshold and pose data are compared to determine whether to switch modes. If the pose data is greater than the pose threshold, the mode is switched to local trajectory correction mode; if the pose data is less than the pose threshold, the mode is switched to hybrid short-term and long-term motion optimization mode.
[0066] In some embodiments, the AI co-driving strategy network model extracts features from the target point image and predicts the target point position and distance through the decoder. Then, by inputting the pose information collected by the IMU and the target point image features, it outputs incremental information of the exoskeleton pose. Based on the incremental pose information, it outputs control commands to the exoskeleton. After the local trajectory correction operation is completed, it can continue to return to the hybrid long-term and short-term motion optimization mode.
[0067] Specifically, when the exoskeleton walks along the trajectory planned by the hybrid long-term and short-term motion optimization mode, it is under passive control. The wearer does not participate in the movement of the exoskeleton, and the local trajectory correction module is not triggered. If an obstacle appears in the walking path and the exoskeleton cannot overcome it, it can only adjust its posture to go around the obstacle. At this time, the wearer can adjust the posture of their upper body, and the corresponding IMU posture will also be adjusted. Then the local trajectory correction mode is triggered. After obstacle avoidance is completed, the exoskeleton can continue walking along the trajectory originally planned by the hybrid long-term and short-term motion optimization module.
[0068] Furthermore, the AI co-driving strategy network model needs to undergo iterative training before actual deployment. First, real-world environmental images of the application scenario and the wearer's actual pose data are collected to generate a test training set. This test training set data is then input into the AI co-driving strategy network model for continuous iterative training, ultimately resulting in a stable AI co-driving strategy network model. In practical use, IMU sensors can be used to monitor and collect the exoskeleton wearer's pose data in real time. Simultaneously, image acquisition devices are used to collect real-time images of the exoskeleton's motion path, extracting features from the real-world images (i.e., target point images) and calculating the target point coordinates and distances. The pose data, target point image feature data, and target point coordinates and distances are then input into the trained AI co-driving strategy network model. Based on the input, the model calculates and outputs the predicted pose data increment, and outputs control commands to the exoskeleton based on this predicted pose data increment.
[0069] In terms of working principle, local trajectory correction belongs to AI co-driving control. As those skilled in the art can understand, pose data contains the wearer's intention information, that is, the wearer can change the pose data through personal will. The local trajectory correction mode is triggered by the person's pose data. Therefore, when the pose data is incrementally predicted through the AI co-driving strategy network model, the person's intention information or state information participates in the movement of the exoskeleton. At the same time, the prediction strategy of the AI co-driving strategy network model is integrated, and local trajectory correction during the movement of the exoskeleton is realized through human-machine shared control. It needs to be used online in real time.
[0070] Furthermore, the hybrid long-short-term motion optimization includes the following steps S121-S123:
[0071] S121: Long-term planning, used to output the optimal global path; the output of long-term planning is calculated based on the SNN network model; the SNN network model includes an encoding layer and a decoding layer, the encoding layer includes multiple convolutional layers and at least one pooling layer, and the decoding layer includes multiple convolutional layers and at least one upsampling layer.
[0072] Specifically, the SNN network model includes an encoding layer and a decoding layer. The encoding layer consists of multiple convolutional layers and at least one pooling layer. Through pooling operations and downsampling, the size of the original input can be reduced, while the multiple convolutional kernels enhance the SNN network model's ability to process complex features. The decoding layer consists of multiple convolutional layers and at least one upsampling layer. Through upsampling operations and skip connections to integrate low-level features, the accuracy of the network can be further improved.
[0073] Furthermore, the input data of the SNN network model is target point information, including the location and direction of the destination, and the surrounding environment image, etc., and the output is the optimal global path from the starting point to the target point. The output of the SNN network model maintains the same resolution and dimension as the input, and the output is a globally optimal path.
[0074] In some embodiments, the loss function of the SNN network model is expressed as the following formula:
[0075]
[0076] Where a, b, and c represent hyperparameters, P represents the predicted path, T represents the true path, L(·) represents the path length, S(·) represents the similarity between the true path and the predicted path, i represents the i-th row of P and T, and j represents the j-th column of P and T.
[0077] S122: Short-term planning, used to output the foot sequence and whole-body motion trajectory of the exoskeleton; the short-term planning includes: prior information of the preset foot placement points, including foot size, foot constraints and foot state definition; outputting a set of the foot sequence based on the foot transfer strategy and iterative updates of the left and right feet; and outputting the whole-body motion trajectory based on forward and inverse kinematics analysis.
[0078] Specifically, short-term planning requires pre-setting prior information about foot placement, including step size, foot constraints, and foot state definitions. Then, by defining foot transfer strategies and iteratively updating the left and right feet, the next foot placement is obtained, resulting in a set of foot sequences. Finally, methods based on forward and inverse kinematics, including but not limited to numerical calculations based on linkage structures and optimized trajectory angle calculations, are used to obtain the whole-body motion trajectory.
[0079] S123: Based on the long-term and short-term plans, generate the three-dimensional motion trajectory of the exoskeleton.
[0080] In some embodiments, the outputs of both long-term and short-term planning can be obtained offline, and the planned trajectories are three-dimensional motion trajectories, not limited to straight-line walking or walking in a single direction. Finally, the path information and trajectory obtained by fusing long-term and short-term motion optimization are pre-set in the exoskeleton software before the exoskeleton begins working.
[0081] In terms of working principle: Hybrid short-term and long-term motion optimization belongs to passive control, that is, the motion mode is driven by the exoskeleton movement, that is, the human intention or state information does not participate in the exoskeleton movement.
[0082] Step S130: Perform multi-task tracking of the direction of the three-dimensional movement of the exoskeleton and the walking trajectory to realize the motion tracking control of the exoskeleton.
[0083] Specifically, a multi-task controller can be used in the task space to track multiple tasks, such as the direction of the exoskeleton's three-dimensional motion and its walking trajectory, through a hierarchical control approach. This enables motion tracking control of the exoskeleton. By rationally allocating resources and optimizing control strategies, multi-task tracking can effectively utilize the exoskeleton's power system and sensor equipment, improving the system's energy efficiency and lifespan. Simultaneously tracking multiple tasks can reduce the impact of a single task failure on the overall stability of the system. Through mutual supplementation and adjustment between tasks, the stability and reliability of the exoskeleton under different working conditions can be improved.
[0084] In some embodiments, multi-task tracking of the direction of three-dimensional movement of the exoskeleton and its walking trajectory includes the following steps S131-S136:
[0085] Step S131: Establish the mapping transformation relationship from joint space to task space.
[0086] Specifically, the motion trajectory, including walking position and direction, can be taken as the tracking task, and a mapping transformation relationship from joint space to task space can be established. In some embodiments, this mapping transformation relationship can be expressed as the following formula:
[0087] x i =f i (q),
[0088] Where, x i The task is represented by q, the joint angle is represented by f. i (q) represents the mapping function from joint space to task space, where i represents the i-th task.
[0089] Step S132: Calculate the task space Jacobian based on the mapping transformation relationship.
[0090] In some embodiments, the task space Jacobian can be calculated as follows:
[0091]
[0092] Among them, J i (q) represents the task space Jacobian, q represents the joint angle, i represents the i-th task, and f i (q) represents the mapping function from joint space to task space. The differential representing the joint angle, This represents the differential of the i-th task.
[0093] Step S133: Calculate the augmented Jacobian matrix based on the task space Jacobian.
[0094] Specifically, the augmented Jacobian matrix is obtained by stacking the aforementioned task space Jacobians. In some embodiments, the calculation of the augmented Jacobian matrix can be expressed as the following formula:
[0095]
[0096] Where i represents the i-th task. Represents the augmented Jacobian matrix, q represents the joint angle, and J represents the joint angle. i (q) represents the task space Jacobi. This represents stacking the task space Jacobi from the first to the i-th.
[0097] Step S134: Calculate the null space projection operator based on the augmented Jacobian matrix.
[0098] The null projection operator can be used to stratify tasks, ensuring that low-priority tasks do not affect high-priority tasks. In some embodiments, the calculation of the null projection operator can be expressed as the following formula:
[0099]
[0100] Where, N i (q) represents the null space projection operator, I represents the identity matrix, and q represents the joint angle. represents the transpose of the aforementioned augmented Jacobian matrix, and T represents the symbol for matrix transpose.
[0101] Step S135: Based on the null projection operator, calculate the decoupled Jacobian matrix and the task speed of hierarchical decoupling.
[0102] In some embodiments, the calculation of the decoupled Jacobian matrix and the task speed of hierarchical decoupling can be expressed as the following formula:
[0103]
[0104] in, J represents the Jacobian of decoupling. i (q) represents the Jacobian space of the task space, N i (q) T denoted by , where represents the transpose of the null projection operator, T represents the sign of the matrix transpose, and V represents the decoupling task speed. This represents the product of the decoupled Jacobian and the joint velocity.
[0105] Step S136: Based on the decoupled Jacobian matrix and the hierarchically decoupled task speed, multi-task tracking is performed by the controller. For example, the controller can be an adaptive controller or a proportional-integral-derivative controller.
[0106] In the AI-based co-driving exoskeleton shared control method provided by this invention, 1) real-time pose data of the exoskeleton wearer is acquired, thereby providing data input for the AI-based co-driving exoskeleton shared control method and laying the foundation for the participation of human intention information or state information in exoskeleton movement; 2) based on the pose data, the local trajectory correction mode and the hybrid long-short-term motion optimization mode are switched in real time. The local trajectory correction mode is completed based on the AI co-driving strategy network model. Thus, the exoskeleton can realize real-time switching between AI co-driving control state and passive control state, realizing omnidirectional walking of the exoskeleton in a complex three-dimensional environment; 3) the direction of the three-dimensional movement and the walking trajectory of the exoskeleton are tracked by multiple tasks to realize the motion tracking control of the exoskeleton. Thus, multi-task tracking can simultaneously consider multiple indicators such as the accuracy and stability of the movement direction and walking trajectory, thereby comprehensively optimizing the overall performance of the exoskeleton. At the same time, multi-task tracking can adjust the movement trajectory and control strategy of the exoskeleton in real time to cope with different environmental and task requirements and improve the system's adaptability to complex environments.
[0107] The above are merely several specific implementations of the AI-based co-driving exoskeleton shared control method of the present invention. Each implementation can be implemented independently or in combination, and the present invention is not intended to limit it. Furthermore, the flowchart of the present invention is merely illustrative, and the execution order between the steps is not limited thereto. The splitting, merging, sequential exchange, and other synchronous or asynchronous execution methods of the steps are all within the protection scope of the present invention.
[0108] See below. Figure 2 , Figure 2 A block diagram of the AI-based co-driving exoskeleton sharing control system of the present invention is shown. The AI-based co-driving exoskeleton sharing control system 200 includes:
[0109] The pose data processing module 210 is used to acquire real-time pose data of the exoskeleton wearer, and to switch between the local trajectory correction module 220 and the hybrid long-term and short-term motion optimization module 230 in real time based on the pose data. The local trajectory correction module 220 includes an AI co-driving strategy network model 221.
[0110] The local trajectory correction module 220 is used to output the increment of the predicted pose data to correct the local motion trajectory.
[0111] Further, see Figure 5 , Figure 5A schematic diagram of the local trajectory correction module of the present invention is shown. In some embodiments of this application, the pose data processing module 210 uses an IMU device, which includes an IMU sensor and an IMU acquisition program. The wearer monitors and acquires pose data in real time by wearing the IMU device. Specifically, the installation position of the IMU device on the human body is shown in [reference needed]. Figure 8 , Figure 8 A schematic diagram of the installation location of the IMU device of the present invention is shown. The local trajectory correction module simultaneously captures real-time images of the actual environment of the exoskeleton's motion path through an image acquisition device (not shown in the figure), extracts features from the real environment image, i.e., the target point image information, and calculates the target point coordinates and distances to summarize the target point information; the pose data and target point information are input into the trained AI co-driving strategy network model; based on the input, the model calculates and outputs the increment of the predicted pose data, and outputs control commands to the exoskeleton based on the increment of the predicted pose data.
[0112] The hybrid long-term and short-term motion optimization module 230 is used to output the three-dimensional motion trajectory of the exoskeleton;
[0113] Further, see Figure 6 , Figure 6 A schematic diagram of the operation of the hybrid long-term and short-term motion optimization module 230 of the present invention is shown. In some embodiments of this application, the hybrid long-term and short-term motion optimization module 230 includes a long-term planning module 231 and a short-term planning module 232.
[0114] The long-term planning module 231 is used to output the optimal global path. Its input data is the target point information, i.e. the destination information, including the location, direction, and surrounding environment image of the destination. After processing by a graph spiking neural network (such as the aforementioned SNN network model), it can output the optimal global path from the starting point to the target point.
[0115] The short-term planning module 232 is used to output the foot sequence and whole-body motion trajectory of the exoskeleton. It requires preset prior information on the foot placement, including foot size, foot constraints, and foot state definition. It inputs the optimal global path, outputs a set of foot sequences based on the foot transfer strategy and iterative updates of the left and right feet, and outputs the whole-body motion trajectory of the exoskeleton based on forward and inverse kinematics analysis.
[0116] Specifically, short-term planning requires pre-setting prior information about foot placement, including step size, foot constraints, and foot state definitions. Then, by defining foot transfer strategies and iteratively updating the left and right feet, the next foot placement is obtained, resulting in a set of foot sequences. Finally, methods based on forward and inverse kinematics, including but not limited to numerical calculations based on linkage structures and optimized trajectory angle calculations, are used to obtain the whole-body motion trajectory.
[0117] S123: Based on the long-term and short-term plans, generate the three-dimensional motion trajectory of the exoskeleton.
[0118] The multi-task controller module 240 is used for multi-task tracking of the direction and walking trajectory of the exoskeleton's three-dimensional motion, thereby realizing the walking tracking control of the exoskeleton.
[0119] Further, see Figure 7 , Figure 7 A schematic diagram of a multitasking controller module according to an embodiment of the present invention is shown. The multitasking controller module 240 includes:
[0120] Task module 241 contains N walking tasks. Each walking task is used to define the specific actions or movement paths that the exoskeleton needs to perform, such as walking tasks under different speeds, directions or environmental conditions.
[0121] The zero-space projection module 242 is used to process and optimize the exoskeleton's motion space, ensuring that no motion conflicts or unnecessary overlaps occur when performing multiple tasks. It can project multiple tasks into the motion space to minimize interference and conflicts.
[0122] The motion decoupling module 243 is used to decompose the complex motion of the exoskeleton into simpler and more manageable sub-motions. This allows for more effective control of different parts of the exoskeleton to meet the needs of different tasks and reduces the complexity and computational cost of motion control.
[0123] The hierarchical motion task module 244 is used to manage and execute each walking task in a hierarchical manner according to priority or logical relationship. This ensures that the exoskeleton system can execute tasks layer by layer according to actual needs, thereby ensuring smooth and efficient overall motion control.
[0124] The controller module 245 is used to integrate and manage the task module 241, the zero-space projection module 242, the motion decoupling module 243, and the hierarchical motion task module 244. The controller module 245 is responsible for receiving input information from the task module and other modules, performing coordination and decision-making between multiple tasks, and finally outputting appropriate control signals to realize the multi-task motion control of the exoskeleton.
[0125] In the AI-co-driving exoskeleton sharing control system 200 of the exemplary embodiment of the present invention, 1) the pose data processing module 210 acquires the real-time pose data of the exoskeleton wearer, thereby providing data input for the AI-co-driving exoskeleton sharing control method and providing a basis for realizing the participation of human intention information or state information in exoskeleton movement; 2) the pose data processing module 210 switches between local trajectory correction mode and hybrid long-short-term motion optimization mode in real time based on pose data. The local trajectory correction mode is completed based on the AI co-driving strategy network model. Thus, the exoskeleton can realize real-time switching between AI co-driving control state and passive control state, realizing omnidirectional walking of the exoskeleton in a complex three-dimensional environment; 3) the multi-task controller module 240 performs multi-task tracking of the direction of the three-dimensional movement and the walking trajectory of the exoskeleton, realizing the motion tracking control of the exoskeleton. Thus, multi-task tracking can simultaneously consider multiple indicators such as the accuracy and stability of the movement direction and walking trajectory, thereby comprehensively optimizing the overall performance of the exoskeleton. At the same time, multi-task tracking can adjust the movement trajectory and control strategy of the exoskeleton in real time to cope with different environmental and task requirements and improve the system's adaptability to complex environments.
[0126] Figure 2 This illustration merely illustrates the AI-based co-driving exoskeleton sharing control system 200 provided by the present invention. Without departing from the inventive concept, the splitting, merging, and addition of modules are all within the scope of protection of this invention. The AI-based co-driving exoskeleton sharing control system 200 provided by the present invention can be implemented by software, hardware, firmware, plugins, and any combination thereof; the present invention is not limited thereto.
[0127] See below. Figure 3 and Figure 4 , Figure 3 A block diagram of the AI-based co-driving exoskeleton shared control device of the present invention is shown. Figure 4 This diagram illustrates the operation of the AI-based co-driving exoskeleton shared control device 300 of the present invention. The AI-based co-driving exoskeleton shared control device 300 includes: an exoskeleton 310, which includes a DC brushless motor 311; as shown... Figure 2 The shared control system 320 shown controls the DC brushless motor 311 to drive the exoskeleton 310 to move.
[0128] When the AI-based co-driving exoskeleton shared control device is working, it first acquires the real-time pose data of the exoskeleton wearer through the IMU device worn on the wearer. Then, based on the acquired pose data, it switches between a local trajectory correction mode and a hybrid long-term and short-term motion optimization mode in real time. Specifically, this includes: setting a preset pose threshold for the wearer; comparing the pose threshold with the pose data; if the pose data is greater than the pose threshold, switching to the local trajectory correction module; if the pose data is less than the pose threshold, switching to the hybrid long-term and short-term motion optimization module.
[0129] The local trajectory correction module is based on the AI co-driving strategy network model. Specifically, the wearer monitors and collects pose data in real time by wearing an IMU device, and captures real-time images of the environment along the exoskeleton's motion path using an image acquisition device. The user-related information and environmental information are then input into the trained AI co-driving strategy network model. Based on the input, the model calculates and outputs the incremental changes in the predicted pose data to correct the local trajectory.
[0130] The hybrid long- and short-term motion optimization module can output the exoskeleton's footstep sequence and whole-body motion trajectory.
[0131] The incremental predicted pose data, along with the exoskeleton's footstep sequence and full-body motion trajectory, are output as task commands. A multi-task controller tracks the exoskeleton's 3D motion direction and trajectory, enabling motion tracking control. This multi-task tracking considers multiple indicators simultaneously, such as the accuracy and stability of the motion direction and trajectory, thus comprehensively optimizing the exoskeleton's overall performance. Furthermore, multi-task tracking allows for real-time adjustments to the exoskeleton's motion trajectory and control strategies to adapt to different environments and task requirements, improving the system's adaptability to complex environments. Finally, the AI-based co-driving exoskeleton shared control device outputs appropriate control signals to control a brushless DC motor, driving the exoskeleton to achieve multi-task motion control.
[0132] This application also provides a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, implement the exoskeleton sharing control method based on AI co-driving as described above.
[0133] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, and includes the aforementioned program code for the exoskeleton sharing control method based on AI co-driving, and the program code may be used by or in conjunction with an instruction execution system, device, or apparatus.
[0134] In summary, this application: 1) acquires real-time pose data of the exoskeleton wearer, thereby providing data input for the AI co-driving exoskeleton shared control method and laying the foundation for realizing the participation of human intention information or state information in exoskeleton movement; 2) based on the pose data, switches between local trajectory correction mode and hybrid long-short-term motion optimization mode in real time. The local trajectory correction mode is completed based on the AI co-driving strategy network model. Thus, the exoskeleton can realize real-time switching between AI co-driving control state and passive control state, realizing omnidirectional walking of the exoskeleton in a complex three-dimensional environment; 3) performs multi-task tracking of the direction and trajectory of the exoskeleton's three-dimensional movement to realize motion tracking control of the exoskeleton. Thus, multi-task tracking can simultaneously consider multiple indicators such as the accuracy and stability of the movement direction and walking trajectory, thereby comprehensively optimizing the overall performance of the exoskeleton. At the same time, multi-task tracking can adjust the exoskeleton's movement trajectory and control strategy in real time to cope with different environmental and task requirements and improve the system's adaptability to complex environments.
[0135] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the appended claims.
Claims
1. An exoskeleton sharing control method based on AI-powered co-driving, characterized in that, include: Acquire real-time pose data of exoskeleton wearers; Based on the pose data, the local trajectory correction mode and the hybrid long-term and short-term motion optimization mode are switched in real time. The local trajectory correction mode is completed based on the AI co-driving strategy network model. Multi-task tracking of the direction and trajectory of the three-dimensional movement of the exoskeleton is performed to achieve motion tracking control of the exoskeleton; The switching between local trajectory correction mode and hybrid long-term and short-term motion optimization mode based on the pose data in real time includes: Preset wearer posture threshold; Compare the pose threshold with the pose data; If the pose data is greater than the pose threshold, switch to local trajectory correction mode; if the pose data is less than the pose threshold, switch to hybrid long-term and short-term motion optimization mode. The hybrid long- and short-term motion optimization includes: Long-term planning is used to output the optimal global path; the output of the long-term planning is calculated based on an SNN network model; the SNN network model includes an encoding layer and a decoding layer, the encoding layer includes multiple convolutional layers and at least one pooling layer, and the decoding layer includes multiple convolutional layers and at least one upsampling layer; Short-term planning is used to output the foot sequence and whole-body motion trajectory of the exoskeleton. The short-term planning includes: pre-defined foot placement information, including foot size, foot constraints, and foot state definition; outputting a set of foot sequences based on foot transfer strategies and iterative updates of left and right feet; and outputting the whole-body motion trajectory based on forward and inverse kinematics analysis. Based on the long-term and short-term plans, the three-dimensional motion trajectory of the exoskeleton is generated.
2. The exoskeleton sharing control method based on AI co-driving as described in claim 1, characterized in that, The AI co-driving strategy network model includes: Obtain image information of the target point; Feature extraction is performed on the image information of the target point, and the coordinates and distance of the target point are calculated; Input the pose data, target point image feature data, and target point coordinates and distance; Based on the input, the increment of the predicted pose data is calculated and output.
3. The exoskeleton sharing control method based on AI co-driving as described in claim 1, characterized in that, The loss function of the SNN network model is expressed by the following formula: ， in, All of these represent hyperparameters. Indicates the predicted path, Indicates the actual path, Indicates the length of the path. The similarity between the actual path and the predicted path is represented by i, where i represents the i-th row of P and T, and j represents the j-th column of P and T.
4. The exoskeleton sharing control method based on AI co-driving as described in claim 1, characterized in that, The multi-task tracking of the direction and trajectory of the exoskeleton's three-dimensional movement includes: Establish a mapping transformation relationship from joint space to task space; Based on the aforementioned mapping transformation relationship, the task space Jacobian is calculated; Calculate the augmented Jacobian matrix based on the task space Jacobian; Based on the augmented Jacobian matrix, the null space projection operator is calculated; Based on the null space projection operator, calculate the decoupled Jacobian matrix and the task speed of hierarchical decoupling; Based on the decoupled Jacobian matrix and the hierarchical decoupled task speed, multi-task tracking is performed by the controller.
5. An AI-based co-driving exoskeleton sharing control system for implementing the method of any one of claims 1 to 4, characterized in that, include: The pose data processing module is used to acquire real-time pose data of the exoskeleton wearer; and to switch between the local trajectory correction module and the hybrid long-term and short-term motion optimization module in real time based on the pose data, wherein the local trajectory correction module includes an AI co-driving strategy network model. The local trajectory correction module is used to output the increment of the predicted pose data to correct the local motion trajectory. The hybrid long-term and short-term motion optimization module is used to output the three-dimensional motion trajectory of the exoskeleton; The multi-task controller module is used to perform multi-task tracking of the direction and walking trajectory of the exoskeleton's three-dimensional movement, thereby realizing the walking tracking control of the exoskeleton.
6. The AI-based co-driving exoskeleton sharing control system as described in claim 5, characterized in that, The pose data processing module includes: IMU sensor: used for pose data acquisition; IMU acquisition program: used for pose data processing and, based on the pose data, switching between the local trajectory correction module and the hybrid long-term and short-term motion optimization module in real time.
7. An exoskeleton sharing control device based on AI-powered co-driving, characterized in that, include: An exoskeleton, the exoskeleton including a DC brushless motor; The AI-based co-driving exoskeleton sharing control system as described in claim 5; The AI-based co-driving exoskeleton sharing control system drives the exoskeleton to move by controlling the DC brushless motor.
8. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, the exoskeleton sharing control method based on AI co-driving as described in any one of claims 1-4 is implemented.