Exoskeleton sharing control method, system and device based on AI co-driving and storage medium

Abstract

The invention provides an exoskeleton sharing control method, system and device based on AI co-driving and a storage medium. The exoskeleton sharing control method based on AI co-driving comprises the steps of obtaining real-time pose data of an exoskeleton wearer; based on the pose data, switching between a local track correction mode and a mixed long and short term motion optimization mode is carried out in real time, and the local track correction mode is completed based on an AI co-driving strategy network model; and performing multi-task tracking on the direction and the walking track of the three-dimensional motion of the exoskeleton to realize motion tracking control of the exoskeleton. By adopting the method provided by the invention, when encountering a sudden obstacle, the initial path can be corrected to the optimal path in combination with the pose data and the target point image feature data, so that the exoskeleton is helped to realize omnidirectional walking in a three-dimensional complex environment.

Description

Technical Field

[0001] The present invention relates to the field of exoskeleton technology, and in particular to an exoskeleton shared control method, system, device and storage medium based on AI co-driving. Background Art

[0002] An exoskeleton is a wearable mechanical device that can provide users with additional energy for limb movement, which is used to improve the wearer's walking, weightlifting or other activities, and help enhance the user's strength, stability and athletic ability. With the development of technologies such as sensors, actuators and artificial intelligence, the athletic ability and intelligence level of exoskeleton robots have been further improved. As an assistive robot, it can help the wearer complete rehabilitation training tasks, daily walking assistance, home walking assistance and other tasks, which to a certain extent alleviates the pressure of nursing staff shortage, reduces high labor costs, and further improves the wearer's quality of life.

[0003] At present, there are a large number of heavy manual laborers and patients with lower limb motor dysfunction in the world, so exoskeletons have a huge market and urgent demand worldwide. However, existing exoskeleton planning and control technologies are mainly used in walking scenes in barrier-free spaces, and the travel path is usually limited to a fixed direction. However, in daily life, walking scenes are usually full of various obstacles, such as furniture and other objects, which pose a challenge to the mobility of exoskeletons.

[0004] Therefore, how to enable the exoskeleton to achieve omnidirectional walking in a three-dimensional complex environment is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0005] In order to overcome the defects of the above-mentioned related technologies, the present invention provides an exoskeleton sharing control method, system, device and storage medium based on AI co-driving, which realizes omnidirectional walking of the exoskeleton in a three-dimensional complex environment through exoskeleton motion optimization and control switching strategy of AI co-driving.

[0006] According to one aspect of the present invention, there is provided an exoskeleton sharing control method based on AI co-driving, comprising:

[0007] Obtain real-time posture data of the exoskeleton wearer;

[0008] Based on the posture data, switching between a local trajectory correction mode and a mixed long- and short-term motion optimization mode is performed in real time, wherein the local trajectory correction mode is completed based on an AI co-driving strategy network model;

[0009] Multi-task tracking is performed on the direction and walking trajectory of the exoskeleton's three-dimensional movement to achieve motion tracking control of the exoskeleton.

[0010] In some embodiments of the present application, the real-time switching between the local trajectory correction mode and the mixed long-term and short-term motion optimization mode based on the posture data includes:

[0011] Preset wearer posture threshold;

[0012] comparing the posture threshold with the posture data;

[0013] If the posture data is greater than the posture threshold, it switches to the local trajectory correction mode; if the posture data is less than the posture threshold, it switches to the mixed long-term and short-term motion optimization mode.

[0014] In some embodiments of the present application, the AI ​​co-driving strategy network model includes:

[0015] Get target point image information;

[0016] Extract features from the target point image information and calculate the target point coordinates and distance;

[0017] Input the posture data, target point image feature data, and target point coordinates and distance;

[0018] Based on the input, the increment of the predicted pose data is calculated and output.

[0019] In some embodiments of the present application, the hybrid long-term and short-term motion optimization includes:

[0020] Long-term planning, used to output an optimal global path; the output result of the long-term planning is calculated based on the SNN network model; the SNN network model includes a coding layer and a decoding layer, the coding layer includes multiple convolutional layers and at least one pooling layer, and the decoding layer includes multiple convolutional layers and at least one upsampling layer;

[0021] Short-term planning is used to output the footstep sequence and whole-body motion trajectory of the exoskeleton; the short-term planning includes: prior information of preset footsteps, including footstep size, footstep constraints and footstep state definition; based on footstep transfer strategy and left and right foot iterative updates, a set of footstep sequences is output; based on forward and inverse kinematics analysis, the whole-body motion trajectory is output;

[0022] Based on the long-term planning and short-term planning, a three-dimensional motion trajectory of the exoskeleton is generated.

[0023] In some embodiments of the present application, the loss function of the SNN network model is expressed as the following formula:

[0024]

[0025] Where a, b, and c all represent hyperparameters, P represents the predicted path, T represents the true path, L(·) represents the length of the path, S(·) represents the similarity between the true path and the predicted path, i represents the i-th row of P and T, and j represents the j-th column of P and T.

[0026] In some embodiments of the present application, the multi-task tracking of the direction and walking trajectory of the exoskeleton's three-dimensional movement includes:

[0027] Establish a mapping transformation relationship from joint space to task space;

[0028] Based on the mapping transformation relationship, calculating the task space Jacobian;

[0029] Based on the task space Jacobian, calculating an augmented Jacobian matrix;

[0030] Based on the augmented Jacobian matrix, calculating a null space projection operator;

[0031] Based on the null space projection operator, calculating the decoupled Jacobian matrix and the hierarchical decoupled task speed;

[0032] Based on the decoupled Jacobian matrix and the hierarchically decoupled task speeds, multi-task tracking is performed through a controller.

[0033] According to another aspect of the present application, an exoskeleton sharing control system based on AI co-driving is also provided, comprising:

[0034] A posture data processing module, used to obtain real-time posture data of the exoskeleton wearer; and used to switch between a local trajectory correction module and a hybrid long-term and short-term motion optimization module in real time based on the posture data, wherein the local trajectory correction module includes an AI co-driving strategy network model;

[0035] The local trajectory correction module is used to output the increment of the predicted posture data to correct the local motion trajectory;

[0036] The hybrid long- and short-term motion optimization module is used to output the three-dimensional motion trajectory of the exoskeleton;

[0037] The multi-task controller module is used to perform multi-task tracking on the direction and walking trajectory of the exoskeleton's three-dimensional movement, thereby realizing walking tracking control of the exoskeleton.

[0038] In some embodiments of the present application, the posture data processing module includes:

[0039] IMU sensor: used for posture data collection;

[0040] IMU acquisition program: used for posture data processing and real-time switching between the local trajectory correction module and the mixed long-term and short-term motion optimization module based on the posture data.

[0041] According to another aspect of the present application, an exoskeleton shared control device based on AI co-driving is also provided, including:

[0042] An exoskeleton, the exoskeleton comprising a brushless DC motor;

[0043] As previously described, a shared control system;

[0044] The shared control system drives the exoskeleton to move by controlling the DC brushless motor.

[0045] According to another aspect of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed by a processor, the exoskeleton sharing control method based on AI co-driving as described above is implemented.

[0046] Compared with the prior art, the advantages of the present invention are:

[0047] The exoskeleton shared control method, system, device and storage medium based on AI co-driving provided by the present invention can combine posture data and environmental image information when encountering sudden obstacles during the movement of the exoskeleton, and use AI to make corrections and predictions to obtain the optimal movement path, thereby helping the exoskeleton to achieve omnidirectional walking in a complex three-dimensional environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

[0049] Figure 1 A flow chart of the exoskeleton sharing control method based on AI co-driving of the present invention is shown.

[0050] Figure 2 A module diagram of the exoskeleton sharing control system based on AI co-driving of the present invention is shown.

[0051] Figure 3 A module diagram of an exoskeleton shared control device based on AI co-driving of the present invention is shown.

[0052] Figure 4 A working schematic diagram of the exoskeleton shared control device based on AI co-driving of the present invention is shown.

[0053] Figure 5 The working schematic diagram of the local trajectory correction module of the present invention is shown.

[0054] Figure 6The working schematic diagram of the hybrid long-term and short-term motion optimization module of the present invention is shown.

[0055] Figure 7 A schematic diagram of a multi-task controller module of the present invention is shown.

[0056] Figure 8 A schematic diagram of the installation position of the IMU device of the present invention is shown. DETAILED DESCRIPTION

[0057] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present invention will be more comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0058] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0059] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined. Therefore, the actual execution order may be changed according to the actual situation.

[0060] Figure 1 The schematic diagram of the flow chart of the exoskeleton sharing control method based on AI co-driving of the present invention is shown. The exoskeleton sharing control method based on AI co-driving provided in the present application comprises the following steps S110 to S130:

[0061] Step S110: Acquire real-time posture data of the exoskeleton wearer.

[0062] Specifically, the position and posture data of the exoskeleton wearer can be monitored and collected with the help of hardware devices such as IMU sensors.

[0063] Step S120: Based on the posture data, the local trajectory correction mode and the mixed long-term and short-term motion optimization mode are switched in real time. The local trajectory correction mode is completed based on the AI ​​co-driving strategy network model.

[0064] Specifically, the IMU acquisition program can be used to process the posture data and switch between the local trajectory correction mode and the mixed long-term and short-term motion optimization mode in real time based on the posture data. The local trajectory correction mode is completed based on the AI ​​co-driving strategy network model. The AI ​​co-driving strategy network model includes: obtaining target point image information; extracting features from the target point image information, and calculating the target point coordinates and distance; inputting posture data, target point image feature data, and target point coordinates and distance; based on the input, calculating and outputting the predicted posture data increment.

[0065] In some embodiments, by presetting the wearer's posture threshold in the IMU acquisition program, and then comparing the posture threshold with the posture data, it is determined whether to switch modes; if the posture data is greater than the posture threshold, it switches to the local trajectory correction mode; if the posture data is less than the posture threshold, it switches to the mixed long-term and short-term motion optimization mode.

[0066] In some embodiments, the AI ​​co-driving strategy network model extracts features from the target point image and predicts the target point position and distance through a decoder, and then outputs incremental information of the exoskeleton posture by inputting the posture information collected by the IMU and the target point information such as the image features of the target point image, and outputs control instructions to the exoskeleton based on the incremental posture information. After the local trajectory correction operation is completed, it can continue to return to the mixed long-term and short-term motion optimization mode.

[0067] Specifically, when the exoskeleton walks along the trajectory planned by the hybrid long-term and short-term hybrid optimization mode, it is passive control at this time, the person does not participate in the movement of the exoskeleton, and the local trajectory correction module is not triggered. If an obstacle appears on the walking path at this time, and the exoskeleton cannot cross the obstacle, it can only adjust the exoskeleton posture and bypass the obstacle. At this time, the wearer can adjust the posture of his upper body, and the corresponding IMU posture will also be adjusted. Then the local trajectory correction mode is triggered. After the obstacle avoidance is completed, the exoskeleton can continue to walk along the trajectory planned by the original hybrid long-term and short-term motion optimization module.

[0068] Furthermore, the AI ​​co-driving strategy network model needs to be iteratively trained before it is actually put into use. First, the real environment image of the application scenario and the real posture data of the wearer are collected to generate a test training set. The test training set data is input into the AI ​​co-driving strategy network model for continuous iterative training, and finally a stable AI co-driving strategy network model is obtained. In actual use, an IMU sensor can be used to monitor and collect the posture data of the exoskeleton wearer in real time. At the same time, the real environment image of the exoskeleton movement path is collected in real time through an image acquisition device, and the real environment image, i.e., the target point image information, is feature extracted, and the target point coordinates and distance are calculated; the posture data, target point image feature data, and target point coordinates and distance are input into the trained AI co-driving strategy network model; based on the input, the model calculates and outputs the increment of the predicted posture data, and outputs control instructions to the exoskeleton based on the increment of the predicted posture data.

[0069] In terms of working principle, local trajectory correction belongs to AI co-driving control. As those skilled in the art can understand, the posture data contains the wearer's intention information, that is, the wearer can change the posture data through personal will. The local trajectory correction mode is triggered by the human posture data. Therefore, when the incremental posture data prediction is performed through the AI ​​co-driving strategy network model, the human's intention information or status information participates in the movement of the exoskeleton. At the same time, the prediction strategy of the AI ​​co-driving strategy network model is integrated to realize local trajectory correction during the exoskeleton movement through human-machine shared control, which needs to be used in real time online.

[0070] Furthermore, the hybrid long-term and short-term motion optimization includes the following steps S121-S123:

[0071] S121: long-term planning, used to output the optimal global path; the output result of the long-term planning is calculated based on the SNN network model; the SNN network model includes a coding layer and a decoding layer, the coding layer includes multiple convolutional layers and at least one pooling layer, and the decoding layer includes multiple convolutional layers and at least one upsampling layer.

[0072] Specifically, the SNN network model includes an encoding layer and a decoding layer. The encoding layer includes multiple convolutional layers and at least one pooling layer. Through pooling operations and downsampling, the size of the original input can be reduced. At the same time, through multi-layer convolution kernels, the ability of the SNN network model to process complex features is enhanced. The decoding layer includes multiple convolutional layers and at least one upsampling layer. Through upsampling operations and skip connections to integrate low-level features, the accuracy of the network can be further improved.

[0073] Furthermore, the input data of the SNN network model is the target point information, including the location, direction, surrounding environment image, etc. of the end point, and the output is the optimal global path from the starting point to the target point. The output of the SNN network model maintains the same resolution and dimension as the input, and the output is a global optimal path.

[0074] In some embodiments, the loss function of the SNN network model is expressed as the following formula:

[0075]

[0076] Where a, b, and c all represent hyperparameters, P represents the predicted path, T represents the true path, L(·) represents the length of the path, S(·) represents the similarity between the true path and the predicted path, i represents the i-th row of P and T, and j represents the j-th column of P and T.

[0077] S122: Short-term planning, used to output the exoskeleton's footstep sequence and whole-body motion trajectory; short-term planning includes: prior information of preset footsteps, including footstep size, footstep constraints, and footstep state definition; based on footstep transfer strategy and left and right foot iterative updates, output a set of the footstep sequences; based on forward and inverse kinematics analysis, output the whole-body motion trajectory.

[0078] Specifically, short-term planning requires the prior information of the footsteps to be preset in advance: including the size of the footsteps, footstep constraints, and footstep state definitions, etc. Then, by defining the footstep transfer strategy and iteratively updating the left and right feet, the next footstep is obtained, thereby obtaining a set of footstep sequences. Then, the forward and inverse kinematics methods include but are not limited to numerical calculations based on the connecting rod structure and optimized trajectory angle calculations to obtain the whole body motion trajectory.

[0079] S123: Generate a three-dimensional motion trajectory of the exoskeleton based on the long-term plan and the short-term plan.

[0080] In some embodiments, the output results of long-term planning and short-term planning can be obtained offline, and the planned trajectory is a three-dimensional motion trajectory, not limited to walking in a straight line or in a single direction. The path information, trajectory, etc. obtained by integrating long-term and short-term motion optimization will be set in advance in the exoskeleton program software before the exoskeleton starts working.

[0081] In terms of working principle: hybrid long- and short-term motion optimization belongs to passive control, that is, the movement mode of human movement is completely driven by the movement of the exoskeleton, that is, the human's intention or state information does not participate in the movement of the exoskeleton.

[0082] Step S130: Perform multi-task tracking on the direction of the exoskeleton's three-dimensional motion and walking trajectory to achieve motion tracking control of the exoskeleton.

[0083] Specifically, the multi-task controller can use the method of hierarchical control tasks in the task space to track multiple tasks such as the direction of the exoskeleton's three-dimensional motion and walking trajectory, thereby completing the motion tracking control of the exoskeleton. By rationally allocating resources and optimizing control strategies, multi-task tracking can effectively utilize the exoskeleton's power system and sensor equipment, improve the system's energy efficiency and service life, and track multiple tasks at the same time to reduce the impact of a single task failure on the overall stability of the system. Through mutual complementation and adjustment between tasks, the stability and reliability of the exoskeleton under different working conditions can be improved.

[0084] In some embodiments, multi-task tracking of the direction of the exoskeleton's three-dimensional motion and the walking trajectory includes the following steps S131 to S136:

[0085] Step S131: Establish a mapping transformation relationship from the joint space to the task space.

[0086] Specifically, the motion trajectory including the walking position and direction can be used as tracking tasks, and a mapping transformation relationship from the joint space to the task space can be established. In some embodiments, this mapping transformation relationship can be expressed as the following formula:

[0087] x i =f i (q),

[0088] Among them, x i represents the task, q represents the joint angle, and f i (q) represents the mapping function from joint space to task space, and i represents the i-th task.

[0089] Step S132: Calculate the task space Jacobian based on the mapping transformation relationship.

[0090] In some embodiments, the calculation of the task space Jacobian can be expressed as follows:

[0091]

[0092] Among them, J i (q) represents the Jacobian of the task space, q represents the joint angle, i represents the i-th task, and f i (q) represents the mapping function from joint space to task space, represents the differential of the joint angle, Represents the differential of the i-th task.

[0093] Step S133: Calculate the augmented Jacobian matrix based on the task space Jacobian.

[0094] Specifically, the augmented Jacobian matrix is ​​obtained by stacking the above-mentioned task space Jacobians. In some embodiments, the calculation of the augmented Jacobian matrix can be expressed as the following formula:

[0095]

[0096] Among them, i represents the i-th task, represents the augmented Jacobian matrix, q represents the joint angle, J i (q) represents the task space Jacobian, Represents stacking the task space Jacobians from the 1st to the i-th.

[0097] Step S134: Calculate the null space projection operator based on the augmented Jacobian matrix.

[0098] Through the null space projection operator, tasks can be layered to achieve the characteristic that low priority tasks do not affect high priority tasks. In some embodiments, the calculation of the null space projection operator can be expressed as the following formula:

[0099]

[0100] Among them, N i (q) represents the null space projection operator, I represents the identity matrix, q represents the joint angle, represents the transpose of the aforementioned augmented Jacobian matrix, and T represents the sign of the matrix transpose.

[0101] Step S135: Based on the null space projection operator, the decoupled Jacobian matrix and the hierarchical decoupled task speed are calculated.

[0102] In some embodiments, the calculation of the decoupled Jacobian matrix and the hierarchical decoupled task speed can be expressed as the following formula:

[0103]

[0104] in, represents the decoupled Jacobian, J i (q) represents the task space Jacobian, N i (q) T represents the transpose of the null space projection operator, T represents the sign of the matrix transpose, V represents the decoupled task speed, Represents the product of the decoupled Jacobian and the joint velocity.

[0105] Step S136: Based on the decoupled Jacobian matrix and the hierarchical decoupled task speeds, a controller is used to perform multi-task tracking. For example, the controller may be an adaptive controller or a proportional integral derivative controller.

[0106] In the exoskeleton sharing control method based on AI co-driving provided by the present invention, 1) the real-time posture data of the exoskeleton wearer is obtained, thereby providing data input for the exoskeleton sharing control method of AI co-driving, and providing a basis for realizing the participation of human intention information or state information in exoskeleton movement; 2) based on the posture data, the local trajectory correction mode and the mixed long-term and short-term motion optimization mode are switched in real time, and the local trajectory correction mode is completed based on the AI ​​co-driving strategy network model, thereby, the exoskeleton can realize real-time switching between the AI ​​co-driving control state and the passive control state, and realize the exoskeleton to realize omnidirectional walking in a three-dimensional complex environment; 3) the direction of the three-dimensional movement and the walking trajectory of the exoskeleton are multi-tasked to realize the motion tracking control of the exoskeleton, thereby, the multi-task tracking can simultaneously consider multiple indicators such as the direction of movement, the accuracy of the walking trajectory, and the stability, so as to comprehensively optimize the overall performance of the exoskeleton, and at the same time, the multi-task tracking can instantly adjust the motion trajectory and control strategy of the exoskeleton to cope with different environments and task requirements, and improve the system's adaptability to complex environments.

[0107] The above are only a number of specific implementations of the exoskeleton sharing control method based on AI co-driving of the present invention. Each implementation can be implemented independently or in combination, and the present invention is not limited to this. Furthermore, the flowchart of the present invention is only schematic, and the execution order between the steps is not limited to this. The splitting, merging, order exchange, and other synchronous or asynchronous execution methods of the steps are all within the scope of protection of the present invention.

[0108] See below Figure 2 , Figure 2 The module diagram of the exoskeleton sharing control system based on AI co-driving of the present invention is shown. The exoskeleton sharing control system based on AI co-driving 200 includes:

[0109] A posture data processing module 210 is used to obtain real-time posture data of the exoskeleton wearer and to switch between the local trajectory correction module 220 and the hybrid long-term and short-term motion optimization module 230 in real time based on the posture data. The local trajectory correction module 220 includes an AI co-driving strategy network model 221;

[0110] A local trajectory correction module 220, used to output the increment of the predicted posture data and perform local motion trajectory correction;

[0111] Further, see Figure 5 , Figure 5The working diagram of the local trajectory correction module of the present invention is shown. In some embodiments of the present application, the posture data processing module 210 uses an IMU device, and the IMU device includes an IMU sensor and an IMU acquisition program. The wearer monitors and collects posture data in real time by wearing the IMU device. Specifically, the installation position of the IMU device on the human body is shown in FIG. Figure 8 , Figure 8 The schematic diagram of the installation position of the IMU device of the present invention is shown. The local trajectory correction module simultaneously captures the real environment image of the exoskeleton motion path in real time through an image acquisition device (not shown in the figure), extracts features from the real environment image, i.e., the target point image information, and calculates the target point coordinates and distance to summarize the target point information; the posture data and target point information are input into the trained AI co-driving strategy network model; the model calculates and outputs the increment of the predicted posture data based on the input, and outputs control instructions to the exoskeleton based on the increment of the predicted posture data.

[0112] A hybrid long-term and short-term motion optimization module 230 is used to output the exoskeleton three-dimensional motion trajectory;

[0113] Further, see Figure 6 , Figure 6 FIG. 2 is a schematic diagram showing the operation of the hybrid long-term and short-term motion optimization module 230 of the present invention. In some embodiments of the present application, the hybrid long-term and short-term motion optimization module 230 includes a long-term planning module 231 and a short-term planning module 232.

[0114] The long-term planning module 231 is used to output the optimal global path. Its input data is the target point information, i.e., the end point information, including the location, direction, surrounding environment image, etc. of the end point. After being processed by a graph pulse neural network (such as the aforementioned SNN network model), it can output the optimal global path from the starting point to the target point.

[0115] The short-term planning module 232 is used to output the footstep sequence and whole-body motion trajectory of the exoskeleton; it requires a priori information of the preset footstep point, including the footstep size, footstep constraints and footstep state definition; inputs the optimal global path, and outputs a set of the footstep sequences based on the footstep transfer strategy and left and right foot iterative updates; based on forward and inverse kinematics analysis, outputs the whole-body motion trajectory of the exoskeleton.

[0116] Specifically, short-term planning requires the prior information of the footsteps to be preset in advance: including the size of the footsteps, footstep constraints, and footstep state definitions, etc. Then, by defining the footstep transfer strategy and iteratively updating the left and right feet, the next footstep is obtained, thereby obtaining a set of footstep sequences. Then, the forward and inverse kinematics methods include but are not limited to numerical calculations based on the connecting rod structure and optimized trajectory angle calculations to obtain the whole body motion trajectory.

[0117] S123: Generate a three-dimensional motion trajectory of the exoskeleton based on the long-term plan and the short-term plan.

[0118] The multi-task controller module 240 is used to perform multi-task tracking on the direction of the three-dimensional movement and the walking trajectory of the exoskeleton to achieve walking tracking control of the exoskeleton.

[0119] Further, see Figure 7 , Figure 7 A schematic diagram of a multi-task controller module according to an embodiment of the present invention is shown. The multi-task controller module 240 includes:

[0120] Task module 241, the task module includes N walking tasks, each walking task is used to define a specific action or motion path that the exoskeleton needs to perform, such as walking tasks under different speeds, directions or environmental conditions;

[0121] A null space projection module 242 for processing and optimizing the motion space of the exoskeleton to ensure that there is no motion conflict or unnecessary overlap when performing multiple tasks. It can project multiple tasks in the motion space to minimize interference and conflict;

[0122] A motion decoupling module 243, which is used to decompose the complex motion of the exoskeleton into simpler and more manageable sub-motions. This can more effectively control different parts of the exoskeleton to meet the needs of different tasks and can reduce the complexity and computational cost of motion control;

[0123] The hierarchical motion task module 244 is used to hierarchically manage and execute various walking tasks according to priorities or logical relationships, which can ensure that the exoskeleton system can execute tasks layer by layer according to actual needs, thereby ensuring smooth and efficient overall motion control;

[0124] The controller module 245 is used to integrate and manage the task module 241, the null space projection module 242, the motion decoupling module 243, and the hierarchical motion task module 244. The controller module 245 is responsible for receiving input information from the task module and other modules, performing coordination and decision-making between multiple tasks, and finally outputting appropriate control signals to realize multi-task motion control of the exoskeleton.

[0125] In the exoskeleton sharing control system 200 based on AI co-driving of the exemplary embodiment of the present invention, 1) the real-time posture data of the exoskeleton wearer is obtained through the posture data processing module 210, thereby providing data input for the exoskeleton sharing control method of AI co-driving, and providing a basis for realizing the participation of human intention information or state information in the exoskeleton movement; 2) the posture data processing module 210 performs real-time switching between the local trajectory correction mode and the mixed long-term and short-term motion optimization mode based on the posture data, and the local trajectory correction mode is completed based on the AI ​​co-driving strategy network model, thereby, the exoskeleton can realize real-time switching between the AI ​​co-driving control state and the passive control state, and realizes the exoskeleton to realize omnidirectional walking in a three-dimensional complex environment; 3) the direction and walking trajectory of the three-dimensional movement of the exoskeleton are multi-tasked by the multi-task controller module 240 to realize the motion tracking control of the exoskeleton, thereby, the multi-task tracking can simultaneously consider multiple indicators such as the motion direction, the accuracy and stability of the walking trajectory, so as to comprehensively optimize the overall performance of the exoskeleton, and at the same time, the multi-task tracking can instantly adjust the motion trajectory and control strategy of the exoskeleton to cope with different environments and task requirements, and improve the system's adaptability to complex environments.

[0126] Figure 2 The exoskeleton sharing control system 200 based on AI co-driving provided by the present invention is only schematically shown. Without violating the concept of the present invention, the splitting, merging, and adding of modules are all within the protection scope of the present invention. The exoskeleton sharing control system 200 based on AI co-driving provided by the present invention can be implemented by software, hardware, firmware, plug-ins, and any combination thereof, and the present invention is not limited thereto.

[0127] See below Figure 3 and Figure 4 , Figure 3 The module diagram of the exoskeleton shared control device based on AI co-driving of the present invention is shown, Figure 4 The schematic diagram of the exoskeleton sharing control device based on AI co-driving of the present invention is shown. The exoskeleton sharing control device based on AI co-driving 300 includes: an exoskeleton 310, and the exoskeleton 311 includes a brushless DC motor 311; Figure 2 The shared control system 320 shown drives the exoskeleton 310 to move by controlling the DC brushless motor 311 .

[0128] When the exoskeleton shared control device based on AI co-driving is working, it first obtains the real-time posture data of the exoskeleton wearer through the IMU device worn by the wearer, and then switches between the local trajectory correction mode and the mixed long-term and short-term motion optimization mode in real time based on the acquired posture data, specifically including: presetting the wearer posture threshold; comparing the posture threshold with the posture data, if the posture data is greater than the posture threshold, switching to the local trajectory correction module; if the posture data is less than the posture threshold, switching to the mixed long-term and short-term motion optimization module.

[0129] The local trajectory correction module is completed based on the AI ​​co-driving strategy network model. Specifically, the wearer wears an IMU device to monitor and collect posture data in real time, and uses an image acquisition device to capture real-time images of the real environment of the exoskeleton's motion path. The human-related and environmental information is input into the trained AI co-driving strategy network model; based on the input, the model calculates and outputs the increment of predicted posture data for local trajectory correction.

[0130] The hybrid long- and short-term motion optimization module can output the exoskeleton's footstep sequence and whole-body motion trajectory.

[0131] The increment of the predicted posture data and the footstep sequence and whole-body motion trajectory of the exoskeleton will be output in the form of task instructions. The multi-task controller will perform multi-task tracking on the direction and walking trajectory of the exoskeleton's three-dimensional motion to achieve exoskeleton motion tracking control. Multi-task tracking can simultaneously consider multiple indicators such as motion direction, walking trajectory accuracy, stability, etc., so as to comprehensively optimize the overall performance of the exoskeleton. At the same time, multi-task tracking can instantly adjust the exoskeleton's motion trajectory and control strategy to cope with different environments and task requirements, and improve the system's adaptability to complex environments. The exoskeleton shared control device based on AI co-driving ultimately outputs a suitable control signal to control the DC brushless motor to drive the exoskeleton to achieve the exoskeleton's multi-task motion control.

[0132] The present application also provides a computer-readable storage medium having computer instructions stored thereon. When the computer instructions are executed by a processor, the exoskeleton sharing control method based on AI co-driving as described above is implemented.

[0133] The computer-readable storage medium may include: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes, and include the aforementioned exoskeleton sharing control method based on AI co-driving program code, and the program code can be used by or in combination with an instruction execution system, device or component.

[0134] In summary, the present application, 1) obtains real-time posture data of the exoskeleton wearer, thereby providing data input for the exoskeleton sharing control method of AI co-driving, and provides a basis for realizing the participation of human intention information or status information in exoskeleton movement; 2) based on the posture data, performs real-time switching between the local trajectory correction mode and the mixed long-term and short-term motion optimization mode, and the local trajectory correction mode is completed based on the AI ​​co-driving strategy network model, thereby, the exoskeleton can realize real-time switching between the AI ​​co-driving control state and the passive control state, and realizes the exoskeleton to realize omnidirectional walking in a three-dimensional complex environment; 3) performs multi-task tracking on the direction and walking trajectory of the exoskeleton's three-dimensional movement, and realizes the motion tracking control of the exoskeleton, thereby, multi-task tracking can simultaneously consider multiple indicators such as the direction of movement, accuracy of the walking trajectory, stability, etc., so as to comprehensively optimize the overall performance of the exoskeleton, and at the same time, multi-task tracking can instantly adjust the motion trajectory and control strategy of the exoskeleton to cope with different environments and task requirements, and improve the system's adaptability to complex environments.

[0135] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present invention are indicated by the appended claims.

Claims

1. An exoskeleton sharing control method based on AI co-driving, characterized in that: include: Obtain real-time posture data of the exoskeleton wearer; Based on the posture data, switching between a local trajectory correction mode and a mixed long- and short-term motion optimization mode is performed in real time, wherein the local trajectory correction mode is completed based on an AI co-driving strategy network model; Multi-task tracking is performed on the direction and walking trajectory of the exoskeleton's three-dimensional movement to achieve motion tracking control of the exoskeleton.

2. The exoskeleton sharing control method based on AI co-driving according to claim 1, characterized in that: The real-time switching between the local trajectory correction mode and the mixed long-term and short-term motion optimization mode based on the posture data includes: Preset wearer posture threshold; comparing the posture threshold with the posture data; If the posture data is greater than the posture threshold, it switches to the local trajectory correction mode; if the posture data is less than the posture threshold, it switches to the mixed long-term and short-term motion optimization mode.

3. The exoskeleton sharing control method based on AI co-driving as claimed in claim 1, characterized in that: The AI ​​shared driving strategy network model includes: Get target point image information; Extract features from the target point image information and calculate the target point coordinates and distance; Input the posture data, target point image feature data, and target point coordinates and distance; Based on the input, the increment of the predicted pose data is calculated and output.

4. The exoskeleton sharing control method based on AI co-driving as claimed in claim 2, characterized in that: The hybrid long-term and short-term motion optimization includes: Long-term planning, used to output an optimal global path; the output result of the long-term planning is calculated based on the SNN network model; the SNN network model includes a coding layer and a decoding layer, the coding layer includes multiple convolutional layers and at least one pooling layer, and the decoding layer includes multiple convolutional layers and at least one upsampling layer; Short-term planning is used to output the footstep sequence and whole-body motion trajectory of the exoskeleton; the short-term planning includes: prior information of preset footsteps, including footstep size, footstep constraints and footstep state definition; based on footstep transfer strategy and left and right foot iterative updates, a set of footstep sequences is output; based on forward and inverse kinematics analysis, the whole-body motion trajectory is output; Based on the long-term planning and short-term planning, a three-dimensional motion trajectory of the exoskeleton is generated.

5. The exoskeleton sharing control method based on AI co-driving as claimed in claim 4, characterized in that: The loss function of the SNN network model is expressed as the following formula: Where a, b, and c all represent hyperparameters, P represents the predicted path, T represents the true path, L(·) represents the length of the path, S(·) represents the similarity between the true path and the predicted path, i represents the i-th row of P and T, and j represents the j-th column of P and T.

6. The exoskeleton sharing control method based on AI co-driving as claimed in claim 1, characterized in that: The multi-task tracking of the direction and walking trajectory of the exoskeleton's three-dimensional motion includes: Establish a mapping transformation relationship from joint space to task space; Based on the mapping transformation relationship, calculating the task space Jacobian; Based on the task space Jacobian, calculating an augmented Jacobian matrix; Based on the augmented Jacobian matrix, calculating a null space projection operator; Based on the null space projection operator, calculating the decoupled Jacobian matrix and the hierarchical decoupled task speed; Based on the decoupled Jacobian matrix and the hierarchically decoupled task speeds, multi-task tracking is performed through a controller.

7. The exoskeleton sharing control system based on AI co-driving is characterized by: include: A posture data processing module, used to obtain real-time posture data of the exoskeleton wearer; and used to switch between a local trajectory correction module and a hybrid long-term and short-term motion optimization module in real time based on the posture data, wherein the local trajectory correction module includes an AI co-driving strategy network model; The local trajectory correction module is used to output the increment of the predicted posture data to correct the local motion trajectory; The hybrid long- and short-term motion optimization module is used to output the three-dimensional motion trajectory of the exoskeleton; The multi-task controller module is used to perform multi-task tracking on the direction and walking trajectory of the exoskeleton's three-dimensional movement, thereby realizing walking tracking control of the exoskeleton.

8. The exoskeleton sharing control system based on AI co-driving as claimed in claim 7, characterized in that: The posture data processing module comprises: IMU sensor: used for posture data collection; IMU acquisition program: used for posture data processing and real-time switching between the local trajectory correction module and the mixed long-term and short-term motion optimization module based on the posture data.

9. Exoskeleton shared control device based on AI co-driving, characterized in that: include: An exoskeleton, the exoskeleton comprising a brushless DC motor; The shared control system as claimed in claim 7; The shared control system drives the exoskeleton to move by controlling the DC brushless motor.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by the processor, the exoskeleton sharing control method based on AI co-driving as described in any one of claims 1-6 is implemented.

Images