Flexible ankle joint exoskeleton control method based on multi-modal large model

Abstract

The invention discloses a flexible ankle joint exoskeleton control method based on a multi-modal large model, and the method comprises the steps: enabling a flexible ankle joint robot to collect the data of a surrounding environment through a visual camera part, carrying out the environment recognition, and enabling an image recognition module to carry out the property classification of a text language through a multi-modal technology; the lower limb master controller carries out operation processing analysis on original data collected by the IMU and the force sensor; the operation main control board is used for processing visual part information and human body actual movement gait information in real time; performing weight distribution on text features representing the environment by using CHATGPT, scheduling together with current gait information, and generating gait parameters at the next moment and scheduling a gait control function; and the main control board completes auxiliary control under real-time interaction of the ankle joint and the environment by executing a control algorithm. According to the scheme, a new-generation exoskeleton robot which is low in cost, light in weight and capable of achieving body intelligence is successfully built.

Description

Technical Field

[0001] The present invention relates to the technical field of exoskeleton perception and control, and in particular to a flexible ankle joint exoskeleton control method based on a multi-modal large model. Background Art

[0002] In recent years, the intelligent elderly care industry has developed rapidly. Active exoskeletons can restore walking for elderly people who are unable to walk, and have great application prospects in medical rehabilitation. There are many kinds of auxiliary structures of exoskeletons, and the difference and evaluation of their auxiliary performance is the exoskeleton control method. Excellent control methods are the prerequisite for achieving accurate assistance for patients, and can also avoid falls caused by improper assistance and excessive assistance. Therefore, accurate control of exoskeletons is the key link to ensure that elderly people can recover their walking.

[0003] Existing exoskeleton technology mainly uses IMU and force sensors to capture human motion posture. IMU can capture the posture angle of the ankle joint to realize the characteristic recognition of human posture. The force sensor can provide the force provided by the active drive device to the human body through feedback to ensure closed-loop control in the predetermined control algorithm. By combining the ankle joint motion posture for zoning and settlement, it finally realizes the characteristic auxiliary control for different sections within a cycle. The control strategy based on IMU and force sensors is essentially the capture of the elderly's movements after they occur. It lacks predictability and is a rough control strategy.

[0004] Defects and shortcomings of the existing technology:

[0005] Insufficient real-time performance: Due to the physical hysteresis unit from data capture and control algorithm to drive module response, the motion control of IMU and force sensor has a delay of 700ms. This directly affects the performance of supporting assisted walking speed, and also means that it is impossible to assist in dealing with emergencies.

[0006] Insufficient interactivity: The sensor-based control method cannot monitor the environment in real time, and cannot achieve corresponding levels of auxiliary control for different auxiliary methods such as going up and down stairs. At the same time, the environmental warning based on traditional image recognition does not have the ability to reason and understand, and cannot understand the potential information from the image to adjust the auxiliary strategy of the lower computer.

[0007] Insufficient intelligence: The traditional gait curve generation algorithm based on the analysis results of electromyographic signals of healthy people has low adaptability, while the currently popular CNN-based gait prediction requires users to train a large amount of data, which is not conducive to low-cost daily use by users. In addition, the trained control curve is not universal and cannot be adapted to the elderly with different body shapes and step lengths. Finally, it is impossible to use the captured user data to iteratively adapt the control method, and the generalization ability is not strong.

[0008] With the continuous development of artificial intelligence, new control methods based on data-driven and reinforcement learning have emerged in recent years. These algorithms can achieve more accurate control of human body posture and real-time response. However, most current models require a large amount of data sets to support them. To achieve personal assistance, a large amount of funds are still needed for adaptive data collection. And there is a lack of perception of the surrounding environment. Therefore, research on low-cost, lightweight exoskeleton robots that can achieve embodied intelligence is one of the future development directions. Existing exoskeleton robots have the following technical problems:

[0009] 1. Lack of environmental perception: Due to the adoption of traditional sensor posture monitoring strategies, the visual sensing module is not used to collect the surrounding environment. And when visual information participates in the regulation of control strategies, there is a lack of relatively mature regulation strategies to guide. This leads to the disadvantages of difficulty in capturing environmental information and difficulty in using environmental data.

[0010] 2. Insufficient generalization ability: The visual processing effect based on the current traditional visual algorithm has strict requirements on hardware resources. At the same time, the traditional algorithm can only calibrate and train data sets for specific scenes. This results in high training costs and cannot cope with various environmental recognitions in a wide range of areas, and cannot meet the requirements of domestic products for a wide range of areas.

[0011] 3. Lack of real-time performance: Since traditional sensor algorithms and control strategies have time-delay units in their design, force assistance has a certain lag, making it impossible to provide emergency assistance for posture measurement in a timely manner in the event of an emergency. Summary of the invention

[0012] In response to the above technical problems, the present invention provides a flexible ankle exoskeleton control method based on a multimodal large model.

[0013] The present invention is implemented by adopting the following technical scheme: a flexible ankle exoskeleton control method based on a multi-modal large model, comprising the following steps:

[0014] Step S1: Start the flexible ankle joint robot and perform a power-on self-test. After the self-test passes, proceed to the next step;

[0015] Step S2: The flexible ankle joint robot uses the visual camera part to collect surrounding environment data and perform environmental recognition, and the image recognition module uses multimodal technology to classify the text language by nature;

[0016] Step S3: The lower limb master control of the flexible ankle joint robot exoskeleton performs calculations, processing and analysis on the raw data collected by the IMU and the force sensor, and converts it into information representing the ankle joint movement gait;

[0017] Step S4: The flexible ankle joint robot uses the computing main control board to process the visual information and the actual gait information of the human body in real time; uses CHATGPT to assign weights to the text features representing the environment, and schedules them together with the current gait information to generate the gait parameters at the next moment and schedule the gait control function;

[0018] Step S5: The ankle joint lower limb movement main control board receives the real-time parameters and operation scheduling strategy transmitted by the operation main control board, and the main control board completes the auxiliary control under the real-time interaction between the ankle joint and the environment by executing the control algorithm.

[0019] Specifically, the flexible ankle joint robot exoskeleton host is worn on the patient, and the six-axis attitude sensor and the tension sensor are fixed together above the patient's metatarsophalangeal joint. The six-axis attitude sensor and the tension sensor are both connected to the exoskeleton host to collect the patient's gait information.

[0020] Specifically, the collected gait information is used to detect the patient's gait phase through a finite state machine to determine the patient's gait state, the patient's gait state is predicted based on the Policy Gradient algorithm, and the patient's gait is controlled by controlling the exoskeleton host. The six-axis attitude sensor uses the Kalman filter algorithm to perform signal noise reduction and smoothing processing on the collected signal; the tension sensor is converted into a digital signal after passing through a fourth-order low-pass Butterworth filter and input into the exoskeleton host.

[0021] Specifically, the patient's gait states include: static state, heel-lifting state, swinging state and ground contact state.

[0022] Specifically, the finite state machine detects the gait phase of the patient and specifically includes:

[0023] Store the position data and angular velocity data of the pitch axis of the six-axis attitude sensor into vectors respectively;

[0024] The position and angular velocity vector data are stored in a sliding window, and the mean and variance of the position data and angular velocity data collected at the current moment are calculated respectively as the basis for judging the gait phase;

[0025] The start and end time points of each gait phase are obtained through finite state machine judgment, and the duration of each gait phase is calculated.

[0026] Specifically, the variance calculation formula of the position data is:

[0027]

[0028] The variance calculation formula of the angular velocity data is:

[0029]

[0030] in, The current six-axis attitude sensor pitch position data is stored in the sliding window, and X(n) is the six-axis attitude sensor pitch position data; The pitch angular velocity data of the six-axis attitude sensor at the current moment is stored in the sliding window, and V(n) is the pitch angular velocity data of the six-axis attitude sensor.

[0031] Specifically, determining the patient's gait state includes:

[0032] Determine the variance S of the position data x And the angular velocity data variance S v Are they all less than the static state threshold? If so, it is determined to be static state, the static state position data X(0) is stored as the static reference position, and the state flag variable State is updated to static state;

[0033] When the state flag variable State is in a stationary state, the angular velocity data V(n) and the angular velocity data variance S v If it is greater than 0, the state flag variable State is updated to the lifting state;

[0034] When the state flag variable State is in the heel-lifting state, the angular velocity data V(n) falls below 0, and the current position data X(n) is greater than the static position reference data X(0), the state flag variable State is updated to the swinging state;

[0035] When the state flag variable State is in the swing state, the angular velocity data V(n) increases to above 0, and the current position data X(n) is less than the static position reference data X(0), the state flag variable State is updated to the touchdown state;

[0036] When the position data variance and the angular velocity data variance decrease to the stationary state threshold, it is determined to be in a stationary state, and the cycle of the finite state machine is completed.

[0037] Specifically, the Policy Gradient algorithm performs extended gradient convergence according to the evaluation value given by the sample entropy value calculation, so that the output assistance size can provide the appropriate amount of force at the appropriate time, and specifically includes the following steps:

[0038] Input the six-axis attitude sensor pitch position data and six-axis attitude sensor pitch angular velocity data, and input the duration of each gait phase;

[0039] Set the state matrix S t for:

[0040]

[0041] Among them, x n and v n They are the position data and angular velocity data of the pitch axis of the six-axis attitude sensor at time n;

[0042] Set the policy function π(F t |s t ; θ0; θ1) is:

[0043]

[0044] Among them, F t is the output motor assist size, s t is the state value at time t, θ0 and θ1 are the dynamically updated network parameters;

[0045] Set the value function Q(s t ,F t )for:

[0046]

[0047] Among them, η is the expected stable value, and is the sample entropy of the duration of the heel-lifting state predicted by the policy network in a gait cycle;

[0048] Update each parameter, the update formula is:

[0049] F t =π(·|s t ;θ0;θ1);

[0050] q t =Q(s t ,F t );

[0051]

[0052] θt+10,1=θt0,1+β·g(F t ,θt0,1);

[0053] in, are θ0 and θ 0,1 The policy function gradient is , β is the learning rate, and the policy gradient network converges according to the evaluation value given by the sample entropy value, and controls the patient's gait by controlling the motor.

[0054] The beneficial effects of the present invention are: the present invention endows the exoskeleton with the ability to perceive the external environment, and with the help of a multimodal pre-trained large model based on CLIP, a lightweight deployment with a local deployment cost of only 800Mb and a zero-sample recognition rate of up to 80% is achieved. The use of advanced technology intelligently improves the efficiency of environmental segmentation and greatly reduces the cost of the visual module.

[0055] By using CHATGPT to process human posture data and environmental semantic text in real time and using force feedback as an evaluation criterion, adaptive gait assistance control is achieved for the elderly with different body shapes and leg lengths. It has a high generalization ability and ensures rehabilitation exercise assistance in different regions, different populations, and different scenarios.

[0056] By adding an EMS electromyographic sensor module, it is possible to sense in advance when muscles are about to contract and relax, thus achieving a true predictive assistance function. It saves precious time for the semantic segmentation of the environment that requires a lot of calculations and the parameter scheduling of the general large model, thus truly achieving real-time performance.

[0057] Finally, we successfully built a new generation of low-cost, lightweight exoskeleton robot capable of embodied intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0059] Figure 1 This is a flow chart of the flexible ankle exoskeleton control algorithm in an embodiment of the present invention;

[0060] Figure 2 This is a schematic diagram of the semantic segmentation principle of the CLIP multimodal large model in an embodiment of the present invention;

[0061] Figure 3 It is the terrain recognition technology route in the embodiment of the present invention;

[0062] Figure 4 This is a schematic diagram of a flexible ankle joint system in an embodiment of the present invention;

[0063] Figure 5 This is a CLIP environment identification flow chart in an embodiment of the present invention;

[0064] Figure 6 The experimental data of ankle joint of human gait movement in the embodiment of the present invention;

[0065] Figure 7 This is a schematic diagram of the driving principle of the lower limb main control circuit in an embodiment of the present invention;

[0066] Figure 8 This is a circuit schematic diagram of the electric control drive main control board in an embodiment of the present invention;

[0067] Fig. 9 This is a schematic diagram of the circuit of the EMS electromyographic signal board in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0069] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0070] The following is combined with Figures 1 to 9 , some embodiments of the present invention are described in detail. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0071] The present invention proposes a flexible ankle exoskeleton control method based on a multi-modal large model. The detailed process is as follows: Figure 1 ): This method extracts environmental information, processes it with a pre-trained image encoder, and matches it with a pre-assembled text encoder to achieve the association between the environment and text information. The attention module is used to enhance the feature expression capability. The text probability weight is calculated through the training method of contrastive learning to achieve the extraction of environmental information. And by calling CHATGPT for natural language understanding, it is adjusted and posture parsed with the basic function parameters of the exoskeleton control end. Gait adaptive auxiliary control adjustment in multiple environments is achieved.

[0072] To achieve the above object, the present invention adopts the following technical solution:

[0073] A flexible ankle exoskeleton control method based on a multimodal large model method comprises the following steps:

[0074] Step 1: Start the flexible ankle joint robot and perform a power-on self-test. If the self-test passes, proceed to step 2;

[0075] Step 2: The flexible ankle joint visual camera collects the surrounding environment and performs environmental recognition. The image recognition module uses multimodal technology to classify the properties of text language, and finally achieves ( Figure 3 )’s technical route effect;

[0076] Step 3: While the camera is analyzing and processing environmental information, the lower limb control of the exoskeleton will also perform calculations, processing and analysis on the raw data collected by the IMU and force sensor, converting it into information representing the ankle joint movement gait.

[0077] Step 4: The flexible ankle joint computing control board processes the visual information and the actual gait information of the human body in real time. Use CHATGPT to assign weights to the text features representing the environment, schedule them together with the current gait information, and finally generate the gait parameters at the next moment and schedule the gait control function.

[0078] Step 5: The ankle joint lower limb motion control board receives the real-time parameters and operation scheduling strategy from the operation control board. The control board completes the auxiliary control of the ankle joint under the real-time interaction with the environment by executing the control algorithm. If a fall is detected based on the body posture analysis transmitted by the IMU during real-time control, proceed to step 6.

[0079] In order to achieve lightweight, low cost and strong generalization, this product adopts the following structural design and biomechanical control algorithm:

[0080] The exoskeleton host is worn on the patient. To ensure the consistency of posture and force closed-loop control direction, the six-axis posture sensor and tension sensor are fixed above the patient's metatarsophalangeal joint. The six-axis posture sensor and tension sensor are connected to the exoskeleton host.

[0081] The patient's gait is divided into four states: static state, heel-lifting state, swinging state, and ground contact state;

[0082] The patient's gait information is collected through a six-axis posture sensor and a tension sensor. The patient's gait phase is detected through a finite state machine to determine the patient's gait. According to the patient's gait state determined by the finite state machine, the patient's gait state is predicted based on the PolicyGradient algorithm, and the patient's gait is controlled by controlling the exoskeleton host.

[0083] In this embodiment, the six-axis attitude sensor uses the Kalman filter algorithm to perform signal noise reduction and smoothing processing on the collected signals; the tension sensor is filtered through a fourth-order low-pass Butterworth filter and then converted into a digital signal to be input into the exoskeleton host.

[0084] In this embodiment, the finite state machine detects the patient's gait phase including the following steps:

[0085] Store the position data and angular velocity data of the pitch axis of the six-axis attitude sensor into vectors respectively;

[0086] The position and angular velocity vector data are stored in a sliding window, and the mean and variance of the position data and angular velocity data collected at the current moment are calculated respectively as the basis for judging the gait phase;

[0087] The start and end time points of each gait phase are obtained through the finite state machine, so as to calculate the duration of each gait phase.

[0088] Preferably, it is characterized in that: the variance of the position data is:

[0089]

[0090] The variance of the angular velocity data is:

[0091]

[0092] in, The current six-axis attitude sensor pitch position data is stored in the sliding window, and X(n) is the six-axis attitude sensor pitch position data; The pitch angular velocity data of the six-axis attitude sensor at the current moment is stored in the sliding window, and V(n) is the pitch angular velocity data of the six-axis attitude sensor.

[0093] In this embodiment, the finite state machine detects the gait phase of the patient, including the following steps:

[0094] Determine the variance S of the position data x And the angular velocity data variance S v Are they all less than the static state threshold? If so, it is determined to be static state, the static state position data X(0) is stored as the static reference position, and the state flag variable State is updated to static state;

[0095] When the state flag variable State is in a stationary state, the angular velocity data V(n) and the angular velocity data variance S v If it is greater than 0, the state flag variable State is updated to the lifting state;

[0096] When the state flag variable State is in the heel-lifting state, the angular velocity data V(n) falls below 0, and the current position data X(n) is greater than the static position reference data X(0), the state flag variable State is updated to the swinging state;

[0097] When the state flag variable State is in the swing state, the angular velocity data V(n) increases to above 0, and the current position data X(n) is less than the static position reference data X(0), the state flag variable State is updated to the touchdown state;

[0098] When the position data variance and the angular velocity data variance decrease to the stationary state threshold, it is determined to be in a stationary state, and the cycle of the finite state machine is completed.

[0099] In this embodiment, the Policy Gradient algorithm performs gradient convergence according to the evaluation value given by the sample entropy value calculation, so that the output assistance size can provide the appropriate amount of force at the appropriate time, including the following steps:

[0100] Input the six-axis attitude sensor pitch position data and six-axis attitude sensor pitch angular velocity data, and input the duration of each gait phase;

[0101] Set the state matrix S t for:

[0102]

[0103] Among them, x n and v n They are the position data and angular velocity data of the pitch axis of the six-axis attitude sensor at time n;

[0104] Set the policy function π(F t |s t ; θ0; θ1) is:

[0105]

[0106] Among them, F t is the output motor assist size, s t is the state value at time t, θ0 and θ1 are the dynamically updated network parameters;

[0107] Set the value function Q(s t ,F t )for:

[0108]

[0109] Among them, η is the expected stable value, and is the sample entropy of the duration of the heel-lifting state predicted by the policy network in a gait cycle;

[0110] Update each parameter, the update formula is:

[0111] F t =π(·|s t ;θ0;θ1);

[0112] q t =Q(s t ,F t );

[0113]

[0114] θt+10,1=θt0,1+β·g(F t ,θt0,1);

[0115] in, are θ0 and θ 0,1 The policy function gradient is , β is the learning rate, and the policy gradient network converges according to the evaluation value given by the sample entropy value, and controls the patient's gait by controlling the motor.

[0116] In one embodiment, the flexible ankle exoskeleton is started and the robot self-check is performed, including: according to the set self-check items, by monitoring the attitude angle of the 9-axis gyroscope and the measurement range of the accelerometer, whether the wearing is correct. At the same time, by recording the rated tension value of the tension sensor, whether the binding is correct, as well as the force value and the straightening force of the mechanical cell climbing line are checked to ensure that the numerical output of the subsequent control method is consistent with the algorithm result. If the status is qualified, the self-check is completed.

[0117] like Figure 5 , is the overall flow chart of the camera obtaining environmental information, through the multi-modal large model ( Figure 2 The CLIP algorithm schematic diagram is used to perform image text matching, including:

[0118] The image is divided into small rectangular areas of fixed size through the ViT image encoder of the CLIP multimodal model, which serves as the basic processing unit of the model.

[0119] The self-attention mechanism calculates the correlation between each image block and all other image blocks, which can capture the global relationship in the image, and perform global relationship modeling and feature extraction for all basic units. The attention module is divided into two parts and three processes. Two parts: self-attention module, cross-attention module, three processes: self-attention module, cross-attention module, self-attention module. The self-attention module enhances the model's own expression ability, and the cross-attention mechanism promotes the information interaction of point cloud features between adjacent fields of view, so that the network has partial perception ability and stronger feature expression ability.

[0120] The final self-attention structure gets the output according to the query, key, value,

[0121]

[0122] Regarding the monitoring of the environment, a text encoder is used for processing. The self-attention mechanism is used to capture the relationship between different words in the sentence. The multi-head mechanism allows the model to understand the correlation between words from different perspectives. Finally, it is converted into a vector through word embedding and the semantic information of the word is captured.

[0123] Text and images are each processed by independent encoders to generate different forms of semantic representations. In terms of text, after passing through the text encoder, the text input is processed into a text vector (Text Embedding), which contains the semantic information of the sentence. Similarly, the image is also processed by the image encoder, and finally an image vector is generated, which captures the visual information of the image. In order to map these two different vector representations to the same dimensional space, the model uses a linear projection layer. The role of this linear layer is to project the vectors of the image and text into the same vector space respectively, so that the semantic similarity can be calculated in the subsequent steps. This process ensures that even if the image and text come from different modalities, they can still be compared in the same space to capture the correlation between them.

[0124] y test =W test ·x test +b test

[0125] y image =W image ·x image +b image

[0126] Through contrastive learning, cosine similarity is used to measure the similarity between image and text representations. The similarity between the image and each text is then calculated using the InfoNCE loss function. The core idea is to maximize the similarity of truly matched image-text pairs and minimize the similarity of random pairs. Under this training method, the model can effectively learn the deep semantic relationship between images and texts, thereby improving cross-modal retrieval or matching capabilities:

[0127]

[0128] Finally, we get the normalized probabilities of the image and the surrounding text, and select the maximum value as the description of the current environment.

[0129] Figure 4 This is a schematic diagram of the flexible ankle joint robot system. The lower limb master controller also performs calculations, processing and analysis on the raw data collected by the IMU and force sensor, and converts it into information representing the ankle joint movement gait, including:

[0130] The IMU is installed on the cuneiform bone of the human foot to obtain the movement posture of the ankle joint. According to the set angle of 29.34°-16.08°, it can confirm that the ankle joint is in plantar flexion or dorsiflexion. Figure 6 ) Human gait movement ankle joint data is then divided into 8 cycles according to the gait curve diagram under a measured gait cycle.

[0131] The force sensor obtains the current force auxiliary value by capturing the tension value of the tension cell climbing line, which is used as the closed-loop feedback value of the PVT force control of the drive module.

[0132] In this embodiment, CHATGPT assigns weights to text features representing the environment, schedules them together with the current gait information, and finally generates gait parameters at the next moment and schedules the gait control function, including:

[0133] Through the interactive method of CHATGPT, the functional parameters of the underlying control function are introduced and an example of scheduling is given. In addition, a feedback adjustment example of the gait PVT curve based on image text is used to schedule function learning of a general large model.

[0134] These include natural language understanding and parsing, which ensures that the robot can understand the target task based on user input; task decomposition and planning, which enables the robot to automatically decompose complex tasks into small tasks and execute them in sequence; robot control interface, which ensures that the robot generates specific actions based on the output of ChatGPT; feedback and learning mechanism, the robot adaptively adjusts its behavior according to real-time methods during task execution, ensuring adaptive control in the environment.

[0135] The lower limb movement control board receives the real-time parameters and operation scheduling strategies from the operation control board and executes the control method in time. Figure 7 As shown, including:

[0136] The main control uses a brushless motor as the driving module for closed-loop control, and uses an embedded chip based on the ST series to perform PID force control through CAN communication. The real-time PVT curve is drawn through the gait curve graph collected by the IMU sensor and biomechanical analysis. It is discretized through the DMP algorithm, and finally a set of 7 PVT value points in 3 dimensions is used. It is convenient for data transmission between the algorithm main control board and the lower limb control main control board, and finally the curve information is restated through the cubic interpolation method to generate a control strategy with denser time periods and faster real-time response in actual control.

[0137] In this embodiment, the implementation of the mechanical structure includes:

[0138] Drive module: In the mechanical body and drive module, the Bowden cable, fixed pulley, and motor are used as the driving part, which can meet 1.2 times the normal walking speed, provide a maximum torque of 62.5NM and a continuous torque of 37.5NM, which can meet more than 50% of the torque required for exercise for the elderly. It has been verified by experiments that it can reduce metabolism by 23%.

[0139] Binding module: In terms of binding, flexible binding is used, which can provide sufficient flexibility to enable the exoskeleton to adapt to the body shape and movement needs of different users, thereby providing more adaptive assistance. At the same time, it reduces friction between the user's body, reduces skin irritation, and prevents potential wear and tear and skin damage. In order to prevent the flexible binding from sliding during exercise, an anti-slip structure is also used in the design to increase friction.

[0140] The framework implementation of the electronic control part includes:

[0141] BSP layer: Initialization includes basic functions such as serial port (one IMU communication, one debug output), buzzer (indicating system status), CAN communication (3508 motor communication), flash (storage data), vofa+ protocol (implementation of host computer protocol) and Debug debugging macro definition

[0142] Peripheral module layer: Currently used peripherals include:

[0143] a. ADS8689: ADC module, reads the analog signal output by the tension sensor, the communication protocol is SPI, and realizes functions such as reset, unipolar conversion, data reading, filtering, etc.;

[0144] b.imu948: IMU sensor, which obtains posture information, determines gait with simulation results, adjusts auxiliary strength, and forms a core closed loop. It realizes functions such as initialization, data reading, Kalman filtering, and gait timing;

[0145] c.M3508: DJI brushless motor, pulls the metal wire to achieve finger flexion and dorsiflexion. It realizes initial tightening, force following feedback output specified torque, overheating and stall protection and other functions.

[0146] Task function layer: The task blocks after decomposing the entire task mainly include:

[0147] a. CAN message receiving and sending: communicate with 3508, check the current, temperature, torque and other information in real time and output;

[0148] b. Gait data processing;

[0149] c. Gait data initialization;

[0150] d.IMU data update and simple filtering derivation;

[0151] e. Gait segmentation, determination of each state (stationary state, heel-lifting state, swinging state, ground contact state), and recording of time;

[0152] f. Time prediction: predict the next state based on the simulation time;

[0153] g. Power assist interface: transmits the predicted data and related parameters to the motor to provide power assist;

[0154] h. Gait data storage: realize basic database interfaces such as adding, deleting, modifying, and averaging;

[0155] i. Communication and curve drawing: communicate with the Linux development board to achieve drawing analysis and other functions.

[0156] Real-time system layer: instantiate a series of functions, accept real-time system scheduling to implement multi-threading:

[0157] a.ADC sampling task: call the ADC interface to achieve real-time data sampling and updating;

[0158] b.CAN communication task: instantiate interrupt function to realize CAN sending and receiving information;

[0159] c. Data processing and communication tasks: communicate with the Linux development board, process and filter sensor data, and upload PVT data packets;

[0160] d. Core control task: call the power assist interface function and assign different scheduling strategies according to the power assist macro definition (divided into 5 levels, 1--full follow mode 2--swing dynamic power assist and other follow-up 3--swing dynamic force control power assist (power assist size 30N) other position control mode 4--full position control mode 5--force PVT curve control);

[0161] e. Equipment initialization task: call each peripheral initialization function, and make sound and light prompts or alarms, and destroy them after the task is completed;

[0162] f.IMU sampling task: call the IMU interface to implement sensor data sampling and update;

[0163] g. Debugging task: Output the real-time data of the system according to the debugging level, and accept the data instructions from the host computer to realize dynamic parameter adjustment.

[0164] Large model interface layer: interface with the large model, implement a series of control and data return APIs for the large model to call:

[0165] a. Communication transceiver: realize initialization, data transceiver, data unpacking and other functions, normally idle, waiting for the large model to be activated;

[0166] b. Adding tasks and message queues: Add data and tasks to the message and task queues respectively according to the unpacked data to implement API calls and message responses.

[0167] Mathematical library dependency: implements a series of mathematical and algorithm-related functions, such as multi-level PID, summation, IMU filtering, Kalman filtering, PVT curve drawing, sliding window filter, data queue, etc.

[0168] a. Add IMU data: add the current IMU angular velocity data to a scrolling window;

[0169] b. Calculate statistics: calculate the variance and mean of the data in the rolling window;

[0170] c. Process the mean data: add the mean data to another rolling window and calculate the difference of these means (equivalent to the slope);

[0171] d. Process the slope data: add the slope data to a new rolling window and calculate the variance of the slope data in the window;

[0172] e. Temporary data processing: perform simple filtering and derivation operations on some data to calculate their variance, mean and difference;

[0173] f. Output test: Contains a commented-out test output statement, which is used to print some key data during debugging.

[0174] The function processes the IMU data through multi-layer rolling windows to perform more complex gait analysis and feature extraction.

[0175] In this embodiment, the framework implementation of the hardware part includes:

[0176] (1) STM32G474 is used as the main control chip. This chip is a mid-to-high-end chip series of STMicroelectronics, with multiple communication interfaces such as serial port, spi, can, etc., and the main frequency is as high as 170MHZ. With hardware floating-point acceleration and real-time operating system, it can effectively handle data transmission and reception, motor closed-loop drive, sensor data processing and other tasks. The PCB board is designed using AltiumDesigner and includes Buck step-down circuit, LDO voltage regulator circuit, main control peripheral circuit, can chip drive circuit and interface circuit, etc. The control board PCB schematic diagram is as follows Fig. 9 Detailed display.

[0177] (2) The electromyography part is modeled after similar foreign products and is designed using Kicad ( Fig. 9) is the actual circuit schematic, which is a pure analog circuit. The front-stage differential op amp uses AD8221 to collect the voltage difference on the electrode sheet; the back-stage amplification and filtering uses TL084 four-way op amp to achieve absolute value, adjustable amplification, filtering, integration and other processing to achieve stable amplification of small signals. The board adopts a miniaturized design and the components are packaged in 0402, which is easy for users to wear and improves comfort.

[0178] For the aforementioned embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily required by the present application.

[0179] The above embodiments describe the basic principles and main features of the present invention and the advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the changes and modifications made by those skilled in the art shall be within the scope of protection of the appended claims of the present invention without departing from the spirit and scope of the present invention.

Claims

1. A flexible ankle exoskeleton control method based on a multimodal large model, characterized in that: The following steps are involved: Step S1: Start the flexible ankle joint robot and perform a power-on self-test. After the self-test passes, proceed to the next step; Step S2: The flexible ankle joint robot uses the visual camera part to collect surrounding environment data and perform environmental recognition, and the image recognition module uses multimodal technology to classify the text language by nature; Step S3: The lower limb master control of the flexible ankle joint robot exoskeleton performs calculations, processing and analysis on the raw data collected by the IMU and the force sensor, and converts it into information representing the ankle joint movement gait; Step S4: The flexible ankle joint robot uses the computing main control board to process the visual information and the actual gait information of the human body in real time; uses CHATGPT to assign weights to the text features representing the environment, and schedules them together with the current gait information to generate the gait parameters at the next moment and schedule the gait control function; Step S5: The ankle joint lower limb movement main control board receives the real-time parameters and operation scheduling strategy transmitted by the operation main control board, and the main control board completes the auxiliary control under the real-time interaction between the ankle joint and the environment by executing the control algorithm.

2. The flexible ankle exoskeleton control method based on a multimodal large model as claimed in claim 1, characterized in that: The flexible ankle joint robot exoskeleton host is worn on the patient, and the six-axis posture sensor and the tension sensor are fixed together above the metatarsophalangeal joint of the patient. The six-axis posture sensor and the tension sensor are both connected to the exoskeleton host to collect the patient's gait information.

3. The flexible ankle exoskeleton control method based on a multimodal large model as claimed in claim 2, characterized in that: The collected gait information is used to detect the patient's gait phase through a finite state machine to determine the patient's gait state, and the patient's gait state is predicted based on the Policy Gradient algorithm. The patient's gait is controlled by controlling the exoskeleton host. The six-axis attitude sensor uses the Kalman filter algorithm to perform signal noise reduction and smoothing processing on the collected signal; the tension sensor is converted into a digital signal after passing through a fourth-order low-pass Butterworth filter and input into the exoskeleton host.

4. The flexible ankle exoskeleton control method based on a multimodal large model as claimed in claim 3, characterized in that: The patient's gait states include: static state, heel-lifting state, swinging state and ground-contacting state.

5. The flexible ankle exoskeleton control method based on a multimodal large model as claimed in claim 4, characterized in that: The finite state machine detects the patient's gait phase specifically including: Store the position data and angular velocity data of the pitch axis of the six-axis attitude sensor into vectors respectively; The position and angular velocity vector data are stored in a sliding window, and the mean and variance of the position data and angular velocity data collected at the current moment are calculated respectively as the basis for judging the gait phase; The start and end time points of each gait phase are obtained through finite state machine judgment, and the duration of each gait phase is calculated.

6. The flexible ankle exoskeleton control method based on a multimodal large model as claimed in claim 5, characterized in that: The variance calculation formula of the position data is: The variance calculation formula of the angular velocity data is: in, The current six-axis attitude sensor pitch position data is stored in the sliding window, and X(n) is the six-axis attitude sensor pitch position data; The pitch angular velocity data of the six-axis attitude sensor at the current moment is stored in the sliding window, and V(n) is the pitch angular velocity data of the six-axis attitude sensor.

7. The flexible ankle exoskeleton control method based on a multimodal large model as claimed in claim 6, characterized in that: Determining the patient's gait state specifically includes: Determine the variance S of the position data x And the angular velocity data variance S v Are they all less than the static state threshold? If so, it is determined to be static state, the static state position data X(0) is stored as the static reference position, and the state flag variable State is updated to static state; When the state flag variable State is in a stationary state, the angular velocity data V(n) and the angular velocity data variance S v If it is greater than 0, the state flag variable State is updated to the lifting state; When the state flag variable State is in the heel-lifting state, the angular velocity data V(n) falls below 0, and the current position data X(n) is greater than the static position reference data X(0), the state flag variable State is updated to the swinging state; When the state flag variable State is in the swing state, the angular velocity data V(n) increases to above 0, and the current position data X(n) is less than the static position reference data X(0), the state flag variable State is updated to the touchdown state; When the position data variance and the angular velocity data variance decrease to the stationary state threshold, it is determined to be in a stationary state, and the cycle of the finite state machine is completed.

8. The flexible ankle exoskeleton control method based on a multimodal large model as claimed in claim 7, characterized in that: The Policy Gradient algorithm performs extended gradient convergence according to the evaluation value given by the sample entropy value calculation, so that the output assistance size can provide the appropriate amount of force at the appropriate time, and specifically includes the following steps: Input the six-axis attitude sensor pitch position data and six-axis attitude sensor pitch angular velocity data, and input the duration of each gait phase; Set the state matrix S t for: Among them, x n and v n They are the position data and angular velocity data of the pitch axis of the six-axis attitude sensor at time n; Set the policy function π(F t |s t ; θ0; θ1) is: Among them, F t is the output motor assist size, s t is the state value at time t, θ0 and θ1 are the dynamically updated network parameters; Set the value function Q(s t , F t )for: Among them, η is the expected stable value, and is the sample entropy of the duration of the heel-lifting state in a gait cycle predicted by the policy network; Update each parameter, the update formula is: F t =π(·|s t ;θ0;θ1); q t =Q(s t ,F t ); θt+10,1=θt0,1+β·g(F t ,θt0,1); in, are θ0 and θ 0，1 The policy function gradient is , β is the learning rate, and the policy gradient network converges according to the evaluation value given by the sample entropy value, and controls the patient's gait by controlling the motor.

Images