Exoskeleton shared control method, system and device based on ai co-driving, and storage medium

Abstract

Provided in the present invention are an exoskeleton shared control method, system and device based on AI co-driving, and a storage medium. The exoskeleton shared control method based on AI co-driving comprises: acquiring real-time pose data of an exoskeleton wearer; on the basis of the pose data, performing switching between a local trajectory correction mode and a hybrid long-short-term motion optimization mode in real time, wherein the local trajectory correction mode is completed on the basis of an AI co-driving policy network model; and performing multi-task tracking on the direction of a three-dimensional motion of an exoskeleton and a walking trajectory thereof, so as to achieve motion tracking control over the exoskeleton. By using the method provided in the present application, when encountering a sudden obstacle, an initial path can be corrected to an optimal path in combination with pose data and target point image feature data, thereby helping an exoskeleton to realize omnidirectional walking in a three-dimensional complex environment.

Description

Exoskeleton shared control method, system, device and storage medium based on AI shared driving Technical Field

[0001] The present invention relates to the field of exoskeleton technology, and in particular to an exoskeleton shared control method, system, device and storage medium based on AI co-driving. Background Art

[0002] An exoskeleton is a wearable mechanical device that provides additional energy for limb movement, improving the wearer's ability to walk, lift weights, or perform other activities, and helping to enhance the wearer's strength, stability, and mobility. With the development of technologies such as sensors, actuators, and artificial intelligence, the mobility and intelligence of exoskeleton robots have been further improved. As an assistive robot, it can help the wearer complete rehabilitation training tasks, assist with daily walking, and assist with walking at home. To a certain extent, it alleviates the pressure of nursing staff shortages, reduces high labor costs, and further improves the wearer's quality of life.

[0003] Currently, there are a large number of people worldwide who do heavy manual labor and suffer from lower limb motor dysfunction. Therefore, there is a huge market and urgent demand for exoskeletons. However, existing exoskeleton planning and control technologies are mainly applied to walking scenarios in barrier-free spaces, and the travel path is usually limited to a fixed direction. However, in daily life, walking scenes are often filled with various obstacles, such as furniture and other objects, which pose a challenge to the mobility of exoskeletons.

[0004] Therefore, how to enable the exoskeleton to achieve omnidirectional walking in a three-dimensional complex environment is a technical problem that needs to be solved urgently in this field.

[0005] Summary of the Invention

[0006] In order to overcome the defects of the above-mentioned related technologies, the present invention provides an exoskeleton sharing control method, system, device and storage medium based on AI co-driving, which realizes omnidirectional walking of the exoskeleton in a complex three-dimensional environment through the exoskeleton motion optimization and control switching strategy of AI co-driving.

[0007] According to one aspect of the present invention, a method for sharing and controlling an exoskeleton based on AI co-driving is provided, comprising:

[0008] Obtain real-time posture data of the exoskeleton wearer;

[0009] Based on the posture data, switching between a local trajectory correction mode and a hybrid long- and short-term motion optimization mode is performed in real time. The local trajectory correction mode is implemented based on the AI ​​co-driving strategy network model.

[0010] Multi-task tracking is performed on the direction of the exoskeleton's three-dimensional movement and walking trajectory to achieve motion tracking control of the exoskeleton.

[0011] In some embodiments of the present application, the real-time switching between the local trajectory correction mode and the hybrid long- and short-term motion optimization mode based on the pose data includes:

[0012] Preset wearer posture threshold;

[0013] comparing the posture threshold with the posture data;

[0014] If the posture data is greater than the posture threshold, the mode is switched to the local trajectory correction mode; if the posture data is less than the posture threshold, the mode is switched to the hybrid long-term and short-term motion optimization mode.

[0015] In some embodiments of the present application, the AI ​​shared driving strategy network model includes:

[0016] Obtain target point image information;

[0017] Extract features from the target point image information and calculate the target point coordinates and distance;

[0018] Input the posture data, target point image feature data, and target point coordinates and distance;

[0019] Based on the input, the increment of the predicted pose data is calculated and output.

[0020] In some embodiments of the present application, the hybrid long-term and short-term motion optimization includes:

[0021] Long-term planning, for outputting an optimal global path; the output result of the long-term planning is calculated based on an SNN network model; the SNN network model includes an encoding layer and a decoding layer, the encoding layer includes multiple convolutional layers and at least one pooling layer, and the decoding layer includes multiple convolutional layers and at least one upsampling layer;

[0022] Short-term planning is used to output the exoskeleton's footstep sequence and full-body motion trajectory. The short-term planning includes: prior information on preset footstep locations, including footstep size, footstep constraints, and footstep state definitions; based on footstep transfer strategies and iterative updates of left and right feet, a set of footstep sequences is output; and based on forward and inverse kinematic analysis, the full-body motion trajectory is output.

[0023] Based on the long-term plan and the short-term plan, a three-dimensional motion trajectory of the exoskeleton is generated.

[0024] In some embodiments of the present application, the loss function of the SNN network model is expressed as the following formula:

[0025] Where a, b, and c are hyperparameters, P is the predicted path, T is the true path, L(·) is the length of the path, S(·) is the similarity between the true path and the predicted path, i is the i-th row of P and T, and j is the j-th column of P and T.

[0026] In some embodiments of the present application, the multi-task tracking of the direction of the exoskeleton's three-dimensional motion and the walking trajectory includes:

[0027] Establish a mapping transformation relationship from joint space to task space;

[0028] Based on the mapping transformation relationship, calculating the task space Jacobian;

[0029] Calculating an augmented Jacobian matrix based on the task space Jacobian;

[0030] Calculating a null space projection operator based on the augmented Jacobian matrix;

[0031] Based on the null space projection operator, calculating the decoupled Jacobian matrix and the hierarchical decoupled task speed;

[0032] Based on the decoupled Jacobian matrix and the hierarchically decoupled task speeds, multi-task tracking is performed through a controller.

[0033] According to another aspect of the present application, an exoskeleton sharing control system based on AI co-driving is also provided, comprising:

[0034] A posture data processing module is used to obtain real-time posture data of the exoskeleton wearer; and is used to switch between a local trajectory correction module and a hybrid long-term and short-term motion optimization module in real time based on the posture data. The local trajectory correction module includes an AI co-driving strategy network model;

[0035] The local trajectory correction module is used to output the increment of the predicted posture data and perform local motion trajectory correction;

[0036] The hybrid long-term and short-term motion optimization module is used to output the three-dimensional motion trajectory of the exoskeleton;

[0037] The multi-task controller module is used to perform multi-task tracking on the direction and walking trajectory of the exoskeleton's three-dimensional movement, thereby realizing walking tracking control of the exoskeleton.

[0038] In some embodiments of the present application, the posture data processing module includes:

[0039] IMU sensor: used for posture data collection;

[0040] IMU acquisition program: used for posture data processing and real-time switching between the local trajectory correction module and the hybrid long-term and short-term motion optimization module based on the posture data.

[0041] According to another aspect of the present application, there is also provided an exoskeleton shared control device based on AI co-driving, comprising:

[0042] An exoskeleton, the exoskeleton comprising a brushless DC motor;

[0043] As previously mentioned, a shared control system;

[0044] The shared control system drives the exoskeleton to move by controlling the brushless DC motor.

[0045] According to another aspect of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed by a processor, the exoskeleton sharing control method based on AI co-driving as described above is implemented.

[0046] Compared with the prior art, the advantages of the present invention are:

[0047] The exoskeleton shared control method, system, device and storage medium based on AI co-driving provided by the present invention can combine posture data and environmental image information when encountering sudden obstacles during the exoskeleton movement, and use AI to make corrections and predictions to obtain the optimal movement path, thereby helping the exoskeleton achieve omnidirectional walking in a complex three-dimensional environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the accompanying drawings.

[0049] FIG1 shows a flow chart of an exoskeleton sharing control method based on AI co-driving according to the present invention.

[0050] FIG2 shows a module diagram of the exoskeleton sharing control system based on AI co-driving of the present invention.

[0051] FIG3 shows a module diagram of an exoskeleton shared control device based on AI co-driving according to the present invention.

[0052] FIG4 shows a schematic diagram of the operation of the exoskeleton shared control device based on AI co-driving of the present invention.

[0053] FIG5 shows a schematic diagram of the operation of the local trajectory correction module of the present invention.

[0054] FIG6 shows a schematic diagram of the operation of the hybrid long-term and short-term motion optimization module of the present invention.

[0055] FIG. 7 shows a schematic diagram of a multi-task controller module according to the present invention.

[0056] FIG8 shows a schematic diagram of the installation position of the IMU device of the present invention. DETAILED DESCRIPTION

[0057] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0058] In addition, the accompanying drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0059] The flowcharts shown in the accompanying drawings are merely exemplary and do not necessarily include all steps. For example, some steps may be decomposed, while some steps may be combined or partially combined. Therefore, the actual execution order may change according to actual circumstances.

[0060] Figure 1 shows a flow chart of the exoskeleton sharing control method based on AI co-driving of the present invention. The exoskeleton sharing control method based on AI co-driving provided by the present application comprises the following steps S110 to S130:

[0061] Step S110: Acquire real-time posture data of the exoskeleton wearer.

[0062] Specifically, the posture data of the exoskeleton wearer can be monitored and collected with the help of hardware devices such as IMU sensors.

[0063] Step S120: Based on the posture data, the local trajectory correction mode and the hybrid long-term and short-term motion optimization mode are switched in real time. The local trajectory correction mode is completed based on the AI ​​co-driving strategy network model.

[0064] Specifically, the IMU acquisition program can be used to process pose data and, based on the pose data, switch between the local trajectory correction mode and the hybrid long- and short-term motion optimization mode in real time. The local trajectory correction mode is completed based on the AI ​​co-driving strategy network model. The AI ​​co-driving strategy network model includes: obtaining target point image information; extracting features from the target point image information, and calculating the target point coordinates and distance; inputting pose data, target point image feature data, and target point coordinates and distance; and based on the input, calculating and outputting the predicted pose data increment.

[0065] In some embodiments, by presetting the wearer's posture threshold in the IMU acquisition program, and then comparing the posture threshold with the posture data, it is determined whether to switch modes; if the posture data is greater than the posture threshold, it switches to the local trajectory correction mode; if the posture data is less than the posture threshold, it switches to the mixed long-term and short-term motion optimization mode.

[0066] In some embodiments, the AI ​​co-driving strategy network model extracts features from the target point image and predicts the target point position and distance through a decoder. It then outputs incremental information of the exoskeleton posture by inputting the posture information collected by the IMU and the target point image features and other target point information. Based on the incremental posture information, it outputs control instructions to the exoskeleton. After the local trajectory correction operation is completed, it can continue to return to the mixed long-term and short-term motion optimization mode.

[0067] Specifically, when the exoskeleton follows a trajectory planned by the hybrid long-term and short-term optimization mode, it is passively controlled; the user does not participate in the exoskeleton's movement, and the local trajectory correction module is not triggered. If an obstacle appears in the walking path and the exoskeleton cannot overcome it, it can only adjust the exoskeleton's posture to bypass the obstacle. At this time, the wearer can adjust the posture of their upper body, and the corresponding IMU posture will also be adjusted. The local trajectory correction mode is then triggered. Once the obstacle avoidance is completed, the exoskeleton can continue walking along the trajectory originally planned by the hybrid long-term and short-term optimization module.

[0068] Furthermore, the AI ​​co-driving strategy network model needs to be iteratively trained before it is actually put into use. First, the real environment image of the application scenario and the wearer's real posture data are collected to generate a test training set. The test training set data is input into the AI ​​co-driving strategy network model for continuous iterative training, and finally a stable AI co-driving strategy network model is obtained. In actual use, an IMU sensor can be used to monitor and collect the posture data of the exoskeleton wearer in real time. At the same time, the image acquisition device can be used to collect real-time environmental images of the exoskeleton's motion path, and feature extraction is performed on the real environment image, i.e., the target point image information, and the target point coordinates and distance are calculated. The posture data, target point image feature data, and target point coordinates and distance are input into the trained AI co-driving strategy network model. Based on the input, the model calculates and outputs the increment of the predicted posture data, and outputs control instructions to the exoskeleton based on the increment of the predicted posture data.

[0069] In terms of working principle, local trajectory correction belongs to AI co-driving control. As those skilled in the art can understand, the posture data contains the wearer's intention information, that is, the wearer can change the posture data through personal will. The local trajectory correction mode is triggered by the human's posture data. Therefore, when the incremental posture data is predicted through the AI ​​co-driving strategy network model, the human's intention information or status information participates in the movement of the exoskeleton. At the same time, the prediction strategy of the AI ​​co-driving strategy network model is integrated to realize local trajectory correction during the exoskeleton movement through human-machine shared control, which needs to be used online in real time.

[0070] Furthermore, the hybrid long-term and short-term motion optimization includes the following steps S121-S123:

[0071] S121: Long-term planning, used to output the optimal global path; the output result of the long-term planning is calculated based on the SNN network model; the SNN network model includes an encoding layer and a decoding layer, the encoding layer includes multiple convolutional layers and at least one pooling layer, and the decoding layer includes multiple convolutional layers and at least one upsampling layer.

[0072] Specifically, the SNN network model includes an encoding layer and a decoding layer. The encoding layer includes multiple convolutional layers and at least one pooling layer. Through pooling operations and downsampling, the size of the original input can be reduced. At the same time, through multiple layers of convolution kernels, the SNN network model's ability to process complex features is enhanced. The decoding layer includes multiple convolutional layers and at least one upsampling layer. Through upsampling operations and skip connections to integrate low-level features, the network's accuracy can be further improved.

[0073] Furthermore, the SNN model takes as input the target point information, including the location, orientation, and surrounding image of the destination, and outputs the optimal global path from the starting point to the destination. The output of the SNN model maintains the same resolution and dimension as the input, and the output is a global optimal path.

[0074] In some embodiments, the loss function of the SNN network model is expressed as the following formula:

[0075] Where a, b, and c are hyperparameters, P is the predicted path, T is the true path, L(·) is the length of the path, S(·) is the similarity between the true path and the predicted path, i is the i-th row of P and T, and j is the j-th column of P and T.

[0076] S122: Short-term planning, used to output the exoskeleton's footstep sequence and whole-body motion trajectory; short-term planning includes: prior information on preset footsteps, including footstep size, footstep constraints, and footstep state definitions; based on footstep transfer strategies and iterative updates of left and right feet, output a set of the footstep sequences; based on forward and inverse kinematic analysis, output the whole-body motion trajectory.

[0077] Specifically, short-term planning requires pre-defined footstep information, including footstep size, footstep constraints, and footstep state definitions. Then, by defining a footstep transfer strategy and iteratively updating the left and right feet, the next footstep is determined, resulting in a set of footstep sequences. Then, forward and inverse kinematics methods, including but not limited to numerical calculations based on linkage structures and optimized trajectory angle calculations, are used to determine the entire body's motion trajectory.

[0078] S123: Generate a three-dimensional motion trajectory of the exoskeleton based on the long-term plan and the short-term plan.

[0079] In some embodiments, both long-term and short-term planning outputs are generated offline, and the planned trajectory is a three-dimensional motion trajectory, not limited to straight lines or single-directional movement. Ultimately, the path information and trajectory obtained through the fusion of long-term and short-term motion optimization are pre-programmed into the exoskeleton software before the exoskeleton begins operation.

[0080] In terms of working principle: hybrid long-term and short-term motion optimization belongs to passive control, that is, the movement mode of human movement is completely driven by the movement of the exoskeleton, that is, the human's intention or state information does not participate in the movement of the exoskeleton.

[0081] Step S130: Perform multi-task tracking on the direction of the exoskeleton's three-dimensional motion and the walking trajectory to achieve motion tracking control of the exoskeleton.

[0082] Specifically, a multi-task controller can track multiple tasks, such as the direction of the exoskeleton's three-dimensional motion and its trajectory, within the task space using a hierarchical control method. This allows for motion tracking control of the exoskeleton. By rationally allocating resources and optimizing control strategies, multi-task tracking can effectively utilize the exoskeleton's power system and sensor equipment, improving the system's energy efficiency and service life. Simultaneously tracking multiple tasks can reduce the impact of a single task failure on the overall system stability. By complementing and regulating tasks, the exoskeleton's stability and reliability can be enhanced under varying operating conditions.

[0083] In some embodiments, multi-task tracking of the direction of the exoskeleton's three-dimensional motion and the walking trajectory includes the following steps S131 to S136:

[0084] Step S131: Establish a mapping transformation relationship from the joint space to the task space.

[0085] Specifically, the motion trajectory including walking position and direction can be used as tracking tasks, and a mapping transformation relationship from joint space to task space can be established. In some embodiments, this mapping transformation relationship can be expressed as the following formula:

[0086] x i =f i (q),

[0087] Among them, x i represents the task, q represents the joint angle, f i (q) represents the mapping function from joint space to task space, and i represents the i-th task.

[0088] Step S132: Calculate the task space Jacobian based on the mapping transformation relationship.

[0089] In some embodiments, the calculation of the task space Jacobian can be expressed as follows:

[0090] Among them, J i (q) represents the task space Jacobian, q represents the joint angle, i represents the i-th task, f i (q) represents the mapping function from joint space to task space, represents the differential of the joint angle, Represents the differential of the i-th task.

[0091] Step S133: Calculate the augmented Jacobian matrix based on the task space Jacobian.

[0092] Specifically, the augmented Jacobian matrix is ​​obtained by stacking the above-mentioned task space Jacobians. In some embodiments, the calculation of the augmented Jacobian matrix can be expressed as the following formula:

[0093] Among them, i represents the i-th task, represents the augmented Jacobian matrix, q represents the joint angle, J i (q) represents the task space Jacobian, Represents stacking the task space Jacobian from the 1st to the i-th.

[0094] Step S134: Calculate a null space projection operator based on the augmented Jacobian matrix.

[0095] By using the null space projection operator, tasks can be layered to ensure that low-priority tasks do not affect high-priority tasks. In some embodiments, the calculation of the null space projection operator can be expressed as the following formula:

[0096] Among them, N i (q) represents the null space projection operator, I represents the identity matrix, q represents the joint angle, represents the transpose of the aforementioned augmented Jacobian matrix, and T represents the symbol of the matrix transpose.

[0097] Step S135: Based on the null space projection operator, the decoupled Jacobian matrix and the hierarchical decoupled task speed are calculated.

[0098] In some embodiments, the calculation of the decoupled Jacobian matrix and the hierarchical decoupled task speed can be expressed as the following formula:

[0099] in, represents the decoupled Jacobian, J i (q) represents the task space Jacobian, N i (q) T represents the transpose of the null space projection operator, T represents the symbol of the matrix transpose, V represents the decoupling task speed, Represents the product of the decoupled Jacobian and the joint velocity.

[0100] Step S136: Based on the decoupled Jacobian matrix and the hierarchically decoupled task speeds, a controller is used to perform multi-task tracking. For example, the controller may be an adaptive controller or a proportional-integral-derivative controller.

[0101] In the exoskeleton sharing control method based on AI co-driving provided by the present invention, 1) real-time posture data of the exoskeleton wearer is obtained, thereby providing data input for the exoskeleton sharing control method of AI co-driving, and providing a basis for realizing the participation of human intention information or status information in exoskeleton movement; 2) based on the posture data, real-time switching between the local trajectory correction mode and the mixed long-term and short-term motion optimization mode is performed, and the local trajectory correction mode is completed based on the AI ​​co-driving strategy network model. Thus, the exoskeleton can realize real-time switching between the AI ​​co-driving control state and the passive control state, and realize omnidirectional walking of the exoskeleton in a three-dimensional complex environment; 3) multi-task tracking is performed on the direction and walking trajectory of the three-dimensional movement of the exoskeleton to realize motion tracking control of the exoskeleton. Thus, multi-task tracking can simultaneously consider multiple indicators such as motion direction, walking trajectory accuracy, stability, etc., thereby comprehensively optimizing the overall performance of the exoskeleton. At the same time, multi-task tracking can instantly adjust the motion trajectory and control strategy of the exoskeleton to cope with different environments and task requirements, thereby improving the system's adaptability to complex environments.

[0102] The above are merely a few specific implementations of the AI-powered shared exoskeleton control method of the present invention. Each implementation can be implemented independently or in combination, and the present invention is not limited thereto. Furthermore, the flowchart of the present invention is merely illustrative, and the order of execution of the steps is not limited thereto. Steps can be split, merged, swapped, or executed synchronously or asynchronously in any other manner, all within the scope of the present invention.

[0103] Referring now to FIG2 , FIG2 shows a block diagram of an AI-based exoskeleton sharing control system of the present invention. The AI-based exoskeleton sharing control system 200 comprises:

[0104] The posture data processing module 210 is used to obtain the real-time posture data of the exoskeleton wearer and to switch between the local trajectory correction module 220 and the hybrid long-term and short-term motion optimization module 230 in real time based on the posture data. The local trajectory correction module 220 includes the AI ​​co-driving strategy network model 221;

[0105] A local trajectory correction module 220 is used to output the increment of the predicted pose data and perform local motion trajectory correction;

[0106] Further, referring to FIG5 , FIG5 shows a schematic diagram of the operation of the local trajectory correction module of the present invention. In some embodiments of the present application, the posture data processing module 210 uses an IMU device, and the IMU device includes an IMU sensor and an IMU acquisition program. The wearer monitors and collects the posture data in real time by wearing the IMU device. Specifically, the installation position of the IMU device on the human body is shown in FIG8 , which shows a schematic diagram of the installation position of the IMU device of the present invention. The local trajectory correction module simultaneously captures the real environment image of the exoskeleton motion path through an image acquisition device (not shown in the figure), extracts features of the real environment image, i.e., the target point image information, and calculates the target point coordinates and distance to summarize the target point information; the posture data and target point information are input into the trained AI co-driving strategy network model; the model calculates and outputs the increment of the predicted posture data based on the input, and outputs control instructions to the exoskeleton based on the increment of the predicted posture data.

[0107] A hybrid long-term and short-term motion optimization module 230 is used to output the exoskeleton's three-dimensional motion trajectory;

[0108] Further, referring to FIG6 , FIG6 shows a schematic diagram of the operation of the hybrid long-term and short-term motion optimization module 230 of the present invention. In some embodiments of the present application, the hybrid long-term and short-term motion optimization module 230 includes a long-term planning module 231 and a short-term planning module 232.

[0109] The long-term planning module 231 is used to output the optimal global path. Its input data is the target point information, i.e., the end point information, including the position, direction, surrounding environment image, etc. of the end point. After being processed by the graph pulse neural network (such as the aforementioned SNN network model), it can output the optimal global path from the starting point to the target point.

[0110] The short-term planning module 232 is used to output the exoskeleton's footstep sequence and whole-body motion trajectory. It requires a priori information on the preset footstep points, including footstep size, footstep constraints, and footstep state definitions. It inputs the optimal global path and outputs a set of the footstep sequences based on the footstep transfer strategy and iterative updates of the left and right feet. It also outputs the exoskeleton's whole-body motion trajectory based on forward and inverse kinematic analysis.

[0111] Specifically, short-term planning requires pre-defined footstep information, including footstep size, footstep constraints, and footstep state definitions. Then, by defining a footstep transfer strategy and iteratively updating the left and right feet, the next footstep is determined, resulting in a set of footstep sequences. Then, forward and inverse kinematics methods, including but not limited to numerical calculations based on linkage structures and optimized trajectory angle calculations, are used to determine the entire body's motion trajectory.

[0112] S123: Generate a three-dimensional motion trajectory of the exoskeleton based on the long-term plan and the short-term plan.

[0113] The multi-task controller module 240 is used to perform multi-task tracking on the direction of the exoskeleton's three-dimensional movement and the walking trajectory, thereby realizing walking tracking control of the exoskeleton.

[0114] Further, referring to FIG7 , FIG7 shows a schematic diagram of a multi-task controller module according to an embodiment of the present invention. The multi-task controller module 240 includes:

[0115] Task module 241, which contains N walking tasks. Each walking task is used to define a specific action or motion path that the exoskeleton needs to perform, such as walking tasks at different speeds, directions, or environmental conditions.

[0116] A null space projection module 242 is used to process and optimize the exoskeleton's motion space to ensure that there is no motion conflict or unnecessary overlap when performing multiple tasks. It can project multiple tasks in the motion space to minimize interference and conflict;

[0117] A motion decoupling module 243 is used to decompose the complex motions of the exoskeleton into simpler, more manageable sub-motions. This allows for more efficient control of different parts of the exoskeleton to meet the needs of different tasks and reduces the complexity and computational cost of motion control;

[0118] Hierarchical motion task module 244 is used to manage and execute walking tasks in layers according to priority or logical relationships, ensuring that the exoskeleton system can execute tasks layer by layer according to actual needs, thereby ensuring smooth and efficient overall motion control;

[0119] The controller module 245 is used to integrate and manage the task module 241, the null space projection module 242, the motion decoupling module 243, and the hierarchical motion task module 244. The controller module 245 is responsible for receiving input information from the task module and other modules, performing coordination and decision-making between multiple tasks, and ultimately outputting appropriate control signals to achieve multi-task motion control of the exoskeleton.

[0120] In an exemplary embodiment of the exoskeleton sharing control system 200 based on AI co-driving, 1) the real-time posture data of the exoskeleton wearer is obtained through the posture data processing module 210, thereby providing data input for the exoskeleton sharing control method for AI co-driving and laying a foundation for implementing the participation of human intention information or state information in the exoskeleton movement. 2) The posture data processing module 210 switches between a local trajectory correction mode and a hybrid long-term and short-term motion optimization mode in real time based on the posture data. The local trajectory correction mode is implemented based on the AI ​​co-driving strategy network model. As a result, the exoskeleton can achieve real-time switching between the AI ​​co-driving control state and the passive control state, enabling the exoskeleton to achieve omnidirectional walking in a three-dimensional complex environment. 3) The multi-task controller module 240 performs multi-task tracking of the direction and walking trajectory of the exoskeleton's three-dimensional movement to achieve motion tracking control of the exoskeleton. As a result, multi-task tracking can simultaneously consider multiple indicators such as motion direction, walking trajectory accuracy, and stability, thereby comprehensively optimizing the overall performance of the exoskeleton. At the same time, multi-task tracking can instantly adjust the exoskeleton's motion trajectory and control strategy to cope with different environments and task requirements, improving the system's adaptability to complex environments.

[0121] Figure 2 is merely a schematic illustration of the AI-powered shared exoskeleton control system 200 provided by the present invention. Without violating the principles of the present invention, the splitting, merging, and adding of modules are all within the scope of the present invention. The AI-powered shared exoskeleton control system 200 provided by the present invention can be implemented using software, hardware, firmware, plug-ins, or any combination thereof, and the present invention is not limited thereto.

[0122] Referring now to Figures 3 and 4 , Figure 3 shows a block diagram of the exoskeleton shared control device for AI co-driving according to the present invention, while Figure 4 shows a schematic diagram of the operation of the exoskeleton shared control device for AI co-driving according to the present invention. Exoskeleton shared control device 300 for AI co-driving includes an exoskeleton 310, which includes a brushless DC motor 311; and a shared control system 320, as shown in Figure 2 , which drives exoskeleton 310 by controlling brushless DC motor 311.

[0123] When the exoskeleton shared control device based on AI co-driving is working, it first obtains the real-time posture data of the exoskeleton wearer through the IMU device worn by the wearer, and then switches between the local trajectory correction mode and the mixed long-term and short-term motion optimization mode in real time based on the acquired posture data. Specifically, it includes: presetting the wearer's posture threshold; comparing the posture threshold with the posture data, if the posture data is greater than the posture threshold, switching to the local trajectory correction module; if the posture data is less than the posture threshold, switching to the mixed long-term and short-term motion optimization module.

[0124] The local trajectory correction module is completed based on the AI ​​co-driving strategy network model. Specifically, the wearer monitors and collects posture data in real time by wearing an IMU device, captures real-time images of the environment of the exoskeleton's motion path through an image acquisition device, and inputs human-related and environmental information into the trained AI co-driving strategy network model; based on the input, the model calculates and outputs the increment of predicted posture data for local trajectory correction.

[0125] The hybrid long-term and short-term motion optimization module can output the exoskeleton's footstep sequence and whole-body motion trajectory.

[0126] The predicted increments of posture data, the exoskeleton's footstep sequence, and the entire body's motion trajectory are output as task instructions. The multi-task controller performs multi-task tracking on the direction and trajectory of the exoskeleton's three-dimensional motion, enabling motion tracking and control of the exoskeleton. Multi-task tracking can simultaneously consider multiple indicators, including motion direction, accuracy of the trajectory, and stability, thereby comprehensively optimizing the exoskeleton's overall performance. Multi-task tracking can also instantly adjust the exoskeleton's motion trajectory and control strategy to accommodate different environments and task requirements, improving the system's adaptability to complex environments. The exoskeleton's shared control device, based on AI co-driving, ultimately outputs appropriate control signals to control the brushless DC motor, driving the exoskeleton to achieve multi-task motion control.

[0127] The present application also provides a computer-readable storage medium having computer instructions stored thereon. When the computer instructions are executed by a processor, the exoskeleton sharing control method based on AI co-driving as described above is implemented.

[0128] The computer-readable storage medium may include: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and includes the aforementioned program code of the exoskeleton sharing control method based on AI co-driving, and the program code can be used by or in combination with an instruction execution system, device or component.

[0129] In summary, the present application, 1) obtains real-time posture data of the exoskeleton wearer, thereby providing data input for the exoskeleton sharing control method of AI co-driving, and provides a basis for realizing the participation of human intention information or status information in exoskeleton movement; 2) based on the posture data, switches between the local trajectory correction mode and the mixed long-term and short-term motion optimization mode in real time, and the local trajectory correction mode is completed based on the AI ​​co-driving strategy network model. Thus, the exoskeleton can realize real-time switching between the AI ​​co-driving control state and the passive control state, and realize omnidirectional walking of the exoskeleton in a three-dimensional complex environment; 3) performs multi-task tracking on the direction and walking trajectory of the exoskeleton's three-dimensional movement to realize motion tracking control of the exoskeleton. Thus, multi-task tracking can simultaneously consider multiple indicators such as motion direction, walking trajectory accuracy, stability, etc., thereby comprehensively optimizing the overall performance of the exoskeleton. At the same time, multi-task tracking can instantly adjust the motion trajectory and control strategy of the exoskeleton to cope with different environments and task requirements, thereby improving the system's adaptability to complex environments.

[0130] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the appended claims.

Claims

1. Exoskeleton shared control method based on AI co-driving, characterized by: include: Obtain real-time posture data of the exoskeleton wearer; Based on the posture data, switching between a local trajectory correction mode and a hybrid long- and short-term motion optimization mode is performed in real time. The local trajectory correction mode is implemented based on the AI ​​co-driving strategy network model. Multi-task tracking is performed on the direction of the exoskeleton's three-dimensional movement and walking trajectory to achieve motion tracking control of the exoskeleton.

2. The exoskeleton sharing control method based on AI co-driving according to claim 1 is characterized in that: The real-time switching between the local trajectory correction mode and the hybrid long- and short-term motion optimization mode based on the posture data includes: Preset wearer posture threshold; comparing the posture threshold with the posture data; If the posture data is greater than the posture threshold, the mode is switched to the local trajectory correction mode; if the posture data is less than the posture threshold, the mode is switched to the hybrid long-term and short-term motion optimization mode.

3. The exoskeleton sharing control method based on AI co-driving according to claim 1, characterized in that: The AI ​​shared driving strategy network model includes: Obtain target point image information; Extract features from the target point image information and calculate the target point coordinates and distance; Input the posture data, target point image feature data, and target point coordinates and distance; Based on the input, the increment of the predicted pose data is calculated and output.

4. The exoskeleton sharing control method based on AI co-driving according to claim 2, characterized in that: The hybrid long-term and short-term motion optimization includes: Long-term planning, for outputting an optimal global path; the output result of the long-term planning is calculated based on an SNN network model; the SNN network model includes an encoding layer and a decoding layer, the encoding layer includes multiple convolutional layers and at least one pooling layer, and the decoding layer includes multiple convolutional layers and at least one upsampling layer; Short-term planning is used to output the exoskeleton's footstep sequence and full-body motion trajectory. The short-term planning includes: prior information on preset footstep locations, including footstep size, footstep constraints, and footstep state definitions; based on footstep transfer strategies and iterative updates of left and right feet, a set of footstep sequences is output; and based on forward and inverse kinematic analysis, the full-body motion trajectory is output. Based on the long-term plan and the short-term plan, a three-dimensional motion trajectory of the exoskeleton is generated.

5. The exoskeleton sharing control method based on AI co-driving according to claim 4 is characterized in that: The loss function of the SNN network model is expressed as the following formula: Where a, b, and c are hyperparameters, P is the predicted path, T is the true path, L(·) is the length of the path, S(·) is the similarity between the true path and the predicted path, i is the i-th row of P and T, and j is the j-th column of P and T.

6. The exoskeleton sharing control method based on AI co-driving according to claim 1, characterized in that: The multi-task tracking of the direction and walking trajectory of the exoskeleton's three-dimensional motion includes: Establish a mapping transformation relationship from joint space to task space; Based on the mapping transformation relationship, calculating the task space Jacobian; Calculating an augmented Jacobian matrix based on the task space Jacobian; Calculating a null space projection operator based on the augmented Jacobian matrix; Based on the null space projection operator, calculating the decoupled Jacobian matrix and the hierarchical decoupled task speed; Based on the decoupled Jacobian matrix and the hierarchically decoupled task speeds, multi-task tracking is performed through a controller.

7. The exoskeleton shared control system based on AI co-driving is characterized by: include: A posture data processing module is used to obtain real-time posture data of the exoskeleton wearer; and is used to switch between a local trajectory correction module and a hybrid long-term and short-term motion optimization module in real time based on the posture data. The local trajectory correction module includes an AI co-driving strategy network model; The local trajectory correction module is used to output the increment of the predicted posture data and perform local motion trajectory correction; The hybrid long-term and short-term motion optimization module is used to output the three-dimensional motion trajectory of the exoskeleton; The multi-task controller module is used to perform multi-task tracking on the direction and walking trajectory of the exoskeleton's three-dimensional movement, thereby realizing walking tracking control of the exoskeleton.

8. The exoskeleton sharing control system based on AI co-driving according to claim 7, characterized in that: The posture data processing module includes: IMU sensor: used for posture data collection; IMU acquisition program: used for posture data processing and real-time switching between the local trajectory correction module and the hybrid long-term and short-term motion optimization module based on the posture data.

9. Exoskeleton shared control device based on AI co-driving, characterized by: include: An exoskeleton, the exoskeleton comprising a brushless DC motor; The shared control system according to claim 7; The shared control system drives the exoskeleton to move by controlling the brushless DC motor.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by the processor, the exoskeleton sharing control method based on AI co-driving as described in any one of claims 1 to 6 is implemented.

Images