Human-exoskeleton interactive virtual simulation platform based on Matlab-OpenSim

Abstract

The invention discloses a human-exoskeleton interactive virtual simulation platform, which adopts a layered architecture design and comprises a user interface layer, a simulation engine layer and a data storage layer. According to the platform, an interactive interface is constructed through App Designer, the platform is realized based on a Matlab-OpenSim joint simulation function, and the functions of model import, control algorithm configuration, real-time visualization and data export are supported. Modularized design is achieved through an incremental development mode, PID, fuzzy PID and a sliding mode control algorithm are combined so that a user can optimize an exoskeleton control strategy by himself / herself, and the balance state of the human body is dynamically analyzed through a mass center calculation and unbalance evaluation module. The platform provides a human motion simulation view and a foot supporting surface change diagram, and man-machine interaction force, muscle force and joint track data can be obtained after simulation. The platform solves the problem that an exoskeleton man-machine interaction virtual simulation platform is lacked in the market, efficient tool support can be provided for exoskeleton structure design and control optimization, and the exoskeleton research and development efficiency is remarkably improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of biomechanical simulation and robot control, and relates to a human-exoskeleton interaction virtual simulation platform based on Matlab-OpenSim, which is suitable for exoskeleton structure design, control strategy optimization and human motion analysis. BACKGROUND

[0002] Exoskeleton robots are widely used in the fields of rehabilitation and human assistance, and their design needs to consider mechanical structure, control algorithm and human-machine interaction characteristics. Traditional design relies on physical prototype testing, which is costly and time-consuming. Existing simulation tools focus on single functions and lack integrated platform support for multidisciplinary collaborative optimization. OpenSim has biomechanical simulation capabilities, but lacks sufficient interface interaction; Matlab has advantages in control algorithm development, but lacks deep integration with biomechanical models. Therefore, there is an urgent need for an exoskeleton virtual simulation platform that can integrate multi-field requirements, support real-time simulation and data visualization. SUMMARY

[0003] The application provides a human-exoskeleton interaction virtual simulation platform based on Matlab-OpenSim, which is a platform for realizing human-exoskeleton interaction virtual simulation and visualization of the simulation process based on the joint simulation function of Matlab-OpenSim. The platform realizes modular design through a layered architecture and incremental development mode.

[0004] The technical solution adopted by the application is as follows:

[0005] A human-exoskeleton interaction virtual simulation platform based on Matlab-OpenSim is designed using a three-layer architecture of "user interface layer-simulation engine layer-data storage layer", which includes:

[0006] User interface layer: an interactive interface built based on App Designer, including parameter setting area, simulation observation window, system status window and data export module. The parameter setting part includes necessary content such as loading model files, motion data and setting control parameters by the user, providing data basis for subsequent simulation; the simulation observation window and system status window are the presentation content during simulation, allowing the user to view the simulation at any time; the data export part can be used or not by the user, and after simulation, joint trajectory tracking data, human-machine interaction force and muscle force are exported;

[0007] Data storage layer: This layer manages model files, control parameters, motion data and simulation data. The model files and motion data are provided by the user, corresponding to OpenSim model files in osim format and motion capture files in sto format. The control parameters are set by the user in the interface and are managed as global data by the data storage layer. The simulation data generated during the simulation process includes the center of mass, extrapolated center of mass, support surface and other content. After receiving the results from the simulation engine layer, this layer can record them for the user to call;

[0008] Simulation engine layer: The core computing module based on OpenSim API, integrating control algorithms, center of mass calculation and imbalance evaluation, biomechanical simulation and visualization engine. The parameters of the control algorithm are defined by the user, who can switch control algorithms to compare trajectory tracking effects. The center of mass calculation and imbalance evaluation require the user-defined foot contact points. The biomechanical simulation part mainly combines OpenSim API and uses the human musculoskeletal model simulation function of OpenSim to obtain accurate human-robot interaction force and muscle force data. The visualization engine includes foot support surface change diagram and human motion simulation view, which is convenient for the user to observe in time.

[0009] The core of the simulation platform is the simulation engine layer, which includes multi-index imbalance evaluation algorithm and biomechanical simulation, responsible for imbalance evaluation and biological simulation in the human-robot interaction process.

[0010] The multi-index imbalance evaluation algorithm first needs to obtain XCoM data and foot dynamic support surface data. The human-robot interaction virtual simulation platform adds an inertial measurement unit (IMU) module to the human musculoskeletal model. IMU, as a combination of sensors, usually includes an accelerometer, a gyroscope and a magnetometer, is used to measure the motion state of an object. The rotation matrix obtained by IMU can be used to calculate the position of each joint of the human body. Combined with the mass distribution of each part of the human body and the exoskeleton, the hybrid center of mass position of human-robot can be obtained. The position of the center of mass of an object is determined by the weighted average of the mass and position of each part. The calculation process is based on the following formula.

[0011] p i =p parent +L i ·rot(θ)

[0012] p mass =αp parent +(1-α)p i

[0013] C=∑(m i ·p mass ) / ∑(m i)

[0014] where p i is the joint coordinate to be solved, p parent is the known parent joint coordinate, L i is the length of the segment connected by the parent joint, rot(θ) is the rotation matrix defined by angle θ, p mass is the centroid coordinate of each segment, and α is the percentage of the length of each segment from the parent joint to the centroid, which can be obtained from GB / T 17245-2004 "Inertial Parameters of the Human Body" for humans, such as α = 47.8% for the upper arm, and needs to be calculated by the individual design for exoskeletons.m i is the mass of each segment, and C is the hybrid centroid position of the human-machine system. Based on the centroid position data and the detected step length, the velocity parameter of the hybrid centroid is obtained using the difference method, which is based on discrete time steps, and the formula is Vcom = ΔPcom / Δt. Then, the extrapolated center of mass (XCoM) position is calculated using the formula , which is used in the multi-index imbalance evaluation algorithm. The XCoM data is used to predict the future CoM position, which can be used to evaluate dynamic balance.

[0015] In the calculation of the dynamic support surface of the foot, the support surface range of the human body is closely related to the contact between the foot and the ground, so the human heel, big toe, and little toe need to be marked, and the triangle formed by the above three positions is used to replace the single foot range. The spatial coordinates of the six foot marker points on the left and right sides are calculated according to the human posture, and then the ground projection coordinates are obtained. The convex hull algorithm in Matlab is used to generate the static support surface. The foot center coordinates can be obtained by averaging the ground projection coordinates of the foot marker points, and the foot movement velocity V foot is calculated by difference during the algorithm running.

[0016]

[0017] The static support surface reflects the basic support range, and the dynamic expansion mechanism is additionally established based on the dynamic characteristics of the human body reflected by the CoM velocity information in the XCoM data. When the foot moves, the support surface is extended in the direction of motion to obtain the dynamic support surface, which provides a buffer space for the adjustment of the center of gravity. This dynamic expansion mechanism can avoid the limitations of the static support surface and better meet the balance requirements of the human body in complex movements, such as the adjustment of foot position when lifting heavy objects or the dynamic change of support range when the human body is walking. The vertices of the static support surface are translated in the direction of V foot τ·V footThe dynamic expansion support area is extended, where τ is a dynamic parameter, the value of which is affected by the speed of the human body, and τ is calculated by the following formula, where baseTau and maxTau are empirical values that can be iteratively set through the platform simulation process. Where baseTau is used to ensure the minimum expansion, and maxTau is used to limit the maximum expansion to avoid evaluation errors caused by excessive expansion.

[0018]

[0019] Based on the obtained extrapolated centroid position and support surface information, a hierarchical imbalance evaluation is realized using an index imbalance evaluation algorithm. The index imbalance evaluation algorithm performs three-level evaluation through four indexes, namely XCoM inclusiveness evaluation, double-foot span evaluation, support area evaluation, and speed direction coordination evaluation. Among them, XCoM inclusiveness evaluation is the key index, and the last three indexes play an auxiliary role in imbalance evaluation.

[0020] XCoM inclusiveness evaluation as a key criterion inherits the core idea of the stability margin (MoS) criterion. MoS is defined as the nearest distance from XCoM to the support surface boundary. A positive value indicates stability, and a negative value indicates potential imbalance. This platform extends the one-dimensional projection of MoS to a two-dimensional space for inclusiveness judgment. It replaces the numerical comparison of XCoM single direction projection with the support surface boundary with a more intuitive spatial position inclusiveness relationship for human dynamic balance evaluation. When XCoM exceeds the support surface boundary, the human body is out of the effective support area, triggering a severe imbalance determination immediately.

[0021] Support area evaluation identifies the risk of insufficient support area by comparing the static support area with the static area threshold. A small area means that the foot contact area with the ground is limited, and the human body is prone to break through the support limit when adjusting the center of gravity, such as standing on one foot or toeing. Double-foot span evaluation monitors the distance between the center points of the left and right feet, i.e., the step width, to determine whether it is below the empirical threshold, reflecting the lateral stability of the lower limbs. When the step width is too narrow, the body's ability to resist tilting decreases, especially when carrying heavy objects, causing the center of gravity to shift, and the body to sway, which can easily lead to lateral imbalance due to insufficient lower limb adjustment range. Speed direction coordination evaluation calculates the cosine similarity between CoM speed and foot center speed, reflecting the consistency and stability of the human body CoM movement speed vcom and foot center point speed vfoot. If the difference between the two speed directions is too large, it proves that the human body movement is not coordinated, which can easily lead to imbalance.

[0022] The specific logic of the multi-index imbalance evaluation algorithm in the running process is as follows: when the XCoM exceeds the limit of the dynamic support surface, it is judged that the human body is out of balance. When the double foot span is too small, the support area is too small or the speed direction difference between the human body center of mass and the foot center point is too large during the movement, the counter i is added by 1, where i represents the number of early warning of the three indicators, when i = 0, it means that there is no early warning state, 0 < i < 3, there is early warning, but it is not out of balance, and when i = 3, it is considered to be in an imbalance state.

[0023] The biomechanics simulation part is realized through the OpenSim API, a bushing force module is added according to the designed exoskeleton structure, the interaction between the human body and the exoskeleton is simulated, the Hill muscle model is applied in the muscle force calculation process, the total muscle activation is minimized as the optimization objective, and the calculation of joint torque to muscle force and muscle activation is completed through a static optimization algorithm.

[0024] During the simulation, the foot support surface change diagram and the human body motion simulation view comprehensively reflect the motion of the human body during the simulation, the former adopts color coding to distinguish the static and expanded support surface areas, and the center of mass motion trajectory and speed vector are dynamically drawn in the window, and the latter reflects the real-time tracking of the exoskeleton through a three-dimensional model, the camera view position is limited through code to avoid the human body simulation animation from leaving the window and affecting observation. After the simulation is completed, the virtual simulation platform outputs the actual angle of the exoskeleton and the actual angle of the human body joint in the image drawing window for comparison, which is convenient for analyzing the tracking algorithm effect.

[0025] Compared with the existing invention, the human-exoskeleton interaction virtual simulation platform of the present application adopts a three-level architecture, including a "user interface layer-simulation engine layer-data storage layer", and has necessary functions such as data loading, control algorithm setting, biomechanics simulation and data output, can complete the calculation and output of human-machine interaction force and human muscle force under the target motion trajectory according to user demand, and is convenient for users to confirm the effect of the exoskeleton designed by them on the human body. It can save funds and time resources for various exoskeleton design work and accelerate exoskeleton research and development. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 The human-exoskeleton interaction virtual simulation platform system architecture provided by the present application is shown in the figure;

[0027] Figure 2 The multi-index imbalance evaluation algorithm flowchart provided by the present application is shown in the figure; DETAILED DESCRIPTION

[0028] Example 1

[0029] As Figure 1The human-exoskeleton interaction virtual simulation platform system architecture shown includes a user interface layer, a simulation engine layer, and a data storage layer. The user interface layer uses App Designer to design the login interface and the main interface. The main interface is divided into a parameter setting area, a simulation observation area, and a data export area. The parameter setting area includes OpenSim model setting and control method setting tags. The former supports file loading, joint mapping (associating the human body and exoskeleton active joints), and foot contact point setting. The latter can select PID, fuzzy PID, and sliding mode control algorithms and adjust parameters. The simulation observation area synchronously displays a 3D exoskeleton tracking view and a 2D foot support surface (including centroid trajectory and velocity vector) through a visualization subsystem, and is equipped with a progress bar and a log window. The data export area connects the data storage layer and provides exoskeleton and human joint angle comparison images and data export functions. The simulation engine layer integrates PID, fuzzy PID, and sliding mode control algorithms for researchers to compare effects. Researchers can modify the platform code to add other control methods. The platform realizes biomechanical simulation by calling OpenSim API and calculates human-robot interaction force and human muscle force. The human-exoskeleton interaction virtual simulation platform also includes a multi-index imbalance evaluation algorithm, defines foot contact points, and generates a dynamic support surface using a convex hull expansion algorithm. It combines centroid projection position and velocity vector, foot support area, and foot span for balance judgment. The data storage layer stores simulation results as CSV files and supports export of muscle force, human-robot interaction force, and joint trajectory data. The core functions in the platform are realized based on Matlab-OpenSim joint simulation, and OpenSim API is called to complete human kinematics, dynamics simulation, and other content.

[0030] The platform development in the embodiment is based on Windows 11 system, OpenSim 4.5 and Matlab R2022b are installed, and the hardware configuration is I5-12500H CPU, 16GB memory and RTX 3050 graphics card. The steps are as follows:

[0031] Step one: preparation of pre-model and motion trajectory file

[0032] The exoskeleton is designed and exported as xml and STL files, the human muscle and skeletal model file is modified to obtain a human-machine coupling model. The human marker point running trajectory is obtained through a motion capture system, and it is converted into human joint motion trajectory using OpenSim.

[0033] Step two: platform use

[0034] Load the human-machine coupling model, load the motion trajectory data on demand. Set the correspondence between the exoskeleton active joints and the human joints, select the control method and set the control parameters, define the foot contact point, and run the simulation. During the simulation process, you can observe the human simulation animation and the imbalance evaluation.

[0035] Step three: data output

[0036] After the human motion simulation is completed, the platform will automatically output the motion trajectory of each joint of the human and the exoskeleton, and output the human-machine interaction force data on demand. You can click the "muscle force calculation and output" button to continue the human muscle activation and muscle force calculation, and complete the simulation.

[0037] For example, the above upper limb exoskeleton simulation and control algorithm comparison process, import osim model and sto motion data, according to the selected model to determine the content of joint mapping table and foot contact point, select PID control method and set PID control parameters (P = 1.0, I = 0.12, D = 0.12). After the simulation, you can observe the joint trajectory tracking in the "image rendering" window. Obtain the corresponding data storage table, perform static optimization process, and obtain the muscle related data table after optimization, including muscle force and muscle activation data. After mean filtering, the muscle force is obtained. In the embodiment, the biceps muscle force filtering situation under two kinds of control parameters P = 1.2, I = 0.3, D = 0.3 and P = 1.0, I = 0.12, D = 0.12, reflects the difference of the exoskeleton on the human body under different control strategies.

Claims

1. A Matlab-OpenSim-based human-exoskeleton interaction virtual simulation platform, characterized in that, The three-layer architecture of "user interface layer-simulation engine layer-data storage layer" is adopted for design, including: The user interface layer: the interactive interface based on App Designer, including parameter setting area, simulation observation window, system state window and data export module; The parameter setting part is loaded by the user model file, motion data and control parameters, etc. to provide data basis for subsequent simulation; The simulation observation window and system state window are the presentation content in the simulation process, which is convenient for users to view the simulation situation at any time; The data export part can be used by the user to decide whether to use or not, and export joint trajectory tracking data, human-computer interaction force and muscle force, etc. after simulation; The data storage layer: this layer manages the model file, control parameters, motion data and simulation data; The model file and motion data are provided by the user, corresponding to the osim format OpenSim model file and sto format motion capture file; The control parameters are set by the user in the interface, and the data storage layer manages them as global data; The simulation data generated in the simulation process includes mass center, extrapolated mass center, support surface and other contents, which can be recorded for user calling after receiving the results of the simulation engine layer; The simulation engine layer: the core computing module based on OpenSim API, integrating control algorithm, mass center calculation and imbalance evaluation, biomechanical simulation and visualization engine part; The parameters of the control algorithm part are defined by the user, and the user can switch the control algorithm to compare the trajectory tracking effect; The mass center calculation and imbalance evaluation part needs to use the foot contact point defined by the user; The biomechanical simulation part mainly combines OpenSim API, based on the human musculoskeletal model simulation function of OpenSim, which can get accurate human-computer interaction force and muscle force data; In the visualization engine part, it includes the foot support surface change diagram and human motion simulation view, which is convenient for users to observe in time.

2. The human-exoskeleton interaction virtual simulation platform according to claim 1, further characterized by, The core of the simulation platform is the simulation engine layer, which includes multi-index imbalance evaluation algorithm and biomechanical simulation, responsible for imbalance evaluation and biological simulation in the human-external skeleton interaction process; The multi-index imbalance evaluation algorithm first needs to obtain XCoM data and foot dynamic support surface data; The human-external skeleton interaction virtual simulation platform adds an inertial measurement unit (IMU) module to the human musculoskeletal model; IMU is a combination of sensors, usually including accelerometer, gyroscope and magnetometer, used to measure the motion state of an object, which can be used to calculate the position of each joint of the human body, combined with the mass distribution of each part of the human body and the external skeleton, to get the hybrid mass center position of human-machine; The mass center of an object is determined by the weighted average of the mass and position of each part, and the calculation process is based on the following formula; p i = p parent + L i • rot(θ) p mass = αp parent + (1 - α)p i C=∑(m i ·p mass ) / ∑(m i ) Where, p i Let p be the joint coordinates to be determined. parent Given the coordinates of the parent joint, L i The length of the part connected to the parent joint, rot(θ) is the rotation matrix defined by angle θ, p mass Here are the coordinates of the center of mass for each body part, and α is the percentage of the length of the body segment relative to the distance from the parent joint to the center of mass for each body part. For the human body, this can be found in GB / T 17245-2004 "Inertial Parameters of Adult Humans". For exoskeletons, the proportion of the center of mass needs to be calculated based on the individual design. i Let C be the mass of each body part, and C be the position of the human-machine hybrid center of mass. Based on the center of mass position data and the detection step size, the velocity parameters of the human-machine hybrid center of mass are obtained using the finite difference method. The finite difference method is based on the discrete time step, and the formula is Vcom = ΔPcom / Δt. Then, through... The location of the extrapolated center of mass (XCoM) is calculated. This XCoM data is used in a multi-index imbalance assessment algorithm. It combines the CoM location and velocity to predict the future CoM location and can be used to assess dynamic equilibrium. In the calculation of the dynamic support surface of the foot, the support surface range of the human body is closely related to the contact between the foot and the ground, so the heel, the big toe and the little toe of the human body are marked, and the triangle formed by the above three positions as the vertices replaces the single foot range. The spatial coordinates of the six foot marking points on the left and right sides are calculated according to the human body posture, and the ground projection coordinates are obtained. The static support surface is generated by applying the convex hull algorithm in Matlab. The foot center coordinates can be obtained by averaging the ground projection coordinates of the foot marking points. The foot movement speed V foot : The static support surface reflects the base support range. On this basis, the dynamic characteristics of the human body embodied by the CoM speed information contained in the XCoM data are considered, and a dynamic expansion mechanism is additionally established. When the foot moves, the support surface is extended in the direction of motion to obtain a dynamic support surface, which reserves a buffer space for the adjustment of the center of gravity. This dynamic expansion mechanism can avoid the limitations of the static support surface and better meet the balance requirements of the human body under complex movements, such as the dynamic change of the support range caused by the adjustment of the foot position when lifting heavy objects or the walking of the human body. The vertex of the static support surface is translated along the V foot direction τ·V foot to dynamically expand the support area, where τ is a dynamic parameter, and its value is affected by the speed of the human body. τ is calculated by the following formula: where baseTau and maxTau are empirical values that can be iteratively set through the simulation process of the platform. baseTau is used to ensure the minimum expansion amount, and maxTau is used to limit the maximum expansion amount to avoid evaluation errors caused by excessive expansion. Based on the obtained extrapolated center of mass position and support surface information, a multi-index imbalance evaluation algorithm is used to realize hierarchical imbalance evaluation. The algorithm uses four indexes for three-level evaluation, including XCoM inclusivity evaluation, double-foot span evaluation, support area evaluation, and speed direction coordination evaluation. The XCoM inclusivity evaluation is the key index, and the last three indexes play an auxiliary role in imbalance evaluation.

3. The human-exoskeleton interaction virtual simulation platform according to claim 1 or 2, further characterized by, As the key criterion, the XCoM inclusivity evaluation inherits the core idea of the Margin of Stability (MoS) criterion. MoS is defined as the nearest distance from XCoM to the support surface boundary. A positive value indicates stability, and a negative value indicates potential imbalance. The human-exoskeleton interactive virtual simulation platform extends the one-dimensional projection of MoS to a two-dimensional space for inclusivity judgment. It replaces the numerical comparison of XCoM single direction projection with the support surface boundary with a more intuitive spatial position inclusivity relationship for the evaluation of human dynamic balance. When XCoM exceeds the support surface boundary, the human body is out of the effective support area, triggering a severe imbalance determination.

4. The human-exoskeleton interaction virtual simulation platform according to claim 3, further characterized by, The support area evaluation identifies the risk of insufficient support area by comparing the static support area with the static area threshold. A small area means that the foot-ground contact area is limited, and the human body is prone to break through the support limit when adjusting the center of gravity, such as standing on one foot or toeing. The double-foot span evaluation monitors the distance between the left and right foot center points, i.e., the step width, to determine whether it is below the empirical threshold, reflecting the lateral stability of the lower limbs. When the step width is too narrow, the human body's anti-tilting ability decreases, especially when carrying heavy objects, causing the center of gravity to shift and the body to sway, which may lead to lateral imbalance due to insufficient lower limb adjustment range. The speed direction coordination evaluation calculates the cosine similarity between the CoM speed and the foot center speed, reflecting the consistency and stability of the human CoM movement speed vcom and the foot center point speed vfoot. If the difference between the two speed directions is too large, it proves that the human motion is uncoordinated, which may lead to imbalance.

5. The human-exoskeleton interaction virtual simulation platform according to claim 1, 2 or 4, further characterized by, The specific logic of the multi-index imbalance evaluation algorithm during operation is as follows: when XCoM exceeds the dynamic support surface limit, the human body is considered to be out of balance. When the double-foot span is too small, the support area is too small, or the speed direction difference between the human center of mass and the foot center point is too large, the counter i will increase by 1. Here, i represents the number of pre-warning that takes effect for the three indexes. When i = 0, there is no pre-warning. When 0 < i < 3, there is a pre-warning, but the body is not yet out of balance. When i = 3, the body is considered to be in an imbalance state.

6. The human-exoskeleton interaction virtual simulation platform of claim 5, further characterized by, The biomechanical simulation part is realized through the OpenSim API. According to the designed exoskeleton structure, a bushing force module is added to simulate the interaction between the human body and the exoskeleton. The Hill muscle model is applied in the muscle force calculation process, with the minimum total muscle activation as the optimization objective. The calculation of joint torque to muscle force and muscle activation is completed through a static optimization algorithm.

Images