Man-machine interaction control method and system for wearable flexible outer limb
Through a variety of sensor technologies and Kalman filtering processing, combined with the symmetry of the human kinematic chain, the accuracy and response problems of flexible external limbs in complex environments are solved, achieving high-precision and fast flexible external limb control.
Patent Information
- Application Number
- CN202411384632.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2024-11-29
AI Technical Summary
The existing symmetrical operation implementation of flexible outer limbs in complex operating environments has problems such as insufficient sensor accuracy, delay in data processing and poor environmental adaptability.
A variety of sensor technologies are used to combine Kalman filtering for data processing and fusion updates. The inertial sensor and Hall sensor capture action data in real time, define the Kalman filtered state vector and observation equation, update the state covariance matrix and state vector, and combine human motion. The symmetry of the learning chain adjusts the position of the flexible outer limb.
Accurate symmetrical operations of flexible outer limbs, improve accuracy and response speed, and enhance environmental adaptability.
Smart Images

Figure CN119036459A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field related to human-computer interaction control, and in particular relates to a human-computer interaction control method and system for a wearable flexible external limb. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] Flexible exo-limb robots have the characteristics of high flexibility and high safety, and are widely used to assist people in completing tasks such as detection, grasping and carrying, rehabilitation and medical treatment.
[0004] Symmetrical tasks can ensure the coordination and synchronization of flexible exolimbs in complex operating environments. Current methods for achieving symmetrical tasks have many limitations, such as insufficient sensor accuracy, data processing delays, and poor environmental adaptability. Summary of the invention
[0005] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a human-computer interaction control method and system for a wearable flexible external limb. The present invention integrates multiple sensor technologies and uses Kalman filtering for data processing and fusion update. It has the advantages of high precision, fast response and good environmental adaptability, and can realize precise and symmetrical operation of flexible external limbs.
[0006] To achieve the above object, the first aspect of the present invention provides a human-computer interaction control method for a wearable flexible external limb, the method comprising:
[0007] By means of inertial sensors arranged at the joints of the flexible external limb, the measurement values of the external limb when performing the task are obtained;
[0008] Define the state vector of the Kalman filter, and obtain the state transfer matrix based on the current measurement value of the inertial sensor and the predicted value at the next moment;
[0009] Establishing the observation equation of Kalman filter, and obtaining the state covariance matrix predicted by Kalman filter based on the state transfer matrix;
[0010] Based on the observation values of the inertial sensors and Hall sensors set at the joints of the flexible external limbs, the state covariance matrix and the state vector are updated to obtain the joint pose of the flexible external limbs;
[0011] The obtained joint posture of the flexible external limb is used as feedback data and combined with the target motion data obtained based on the symmetry of the human kinematic chain to adjust the posture of the flexible external limb.
[0012] A second aspect of the present invention provides a wearable flexible external limb human-computer interaction control system, the system comprising:
[0013] An acquisition module, used for acquiring measurement values of the external limb when performing a task through inertial sensors arranged at the joints of the flexible external limb;
[0014] The calculation module is used to define the state vector of the Kalman filter and obtain the state transfer matrix based on the current measurement value of the inertial sensor and the predicted value at the next moment;
[0015] A prediction module, used to establish an observation equation of a Kalman filter, and obtain a state covariance matrix predicted by the Kalman filter based on the state transfer matrix;
[0016] An updating module is used to update the state covariance matrix and the state vector based on the observation values of the inertial sensor and the Hall sensor arranged at the joint of the flexible external limb to obtain the joint posture of the flexible external limb;
[0017] The adjustment module is used to use the obtained joint posture of the flexible external limb as feedback data, combined with the target action data obtained based on the symmetry of the human kinematic chain, to adjust the posture of the flexible external limb.
[0018] The third aspect of the present invention provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, a human-computer interaction control method for a wearable flexible external limb is performed.
[0019] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, executes a human-computer interaction control method for a wearable flexible external limb.
[0020] One or more of the above technical solutions have the following beneficial effects:
[0021] In the present invention, the motion data captured in real time by inertial sensors and Hall sensors are processed based on Kalman filtering to obtain the accurate position and posture of the flexible external limbs; the target motion data obtained based on the symmetry of the human kinematic chain is combined with the obtained accurate position and posture of the flexible external limbs to adjust the flexible external limbs; the present invention integrates multiple sensor technologies and Kalman filtering technology, has the advantages of high precision, fast response and good environmental adaptability, and can realize precise and symmetrical operation of the flexible external limbs.
[0022] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0024] Figure 1 This is a flow chart of the human-computer interaction control method in Embodiment 1 of the present invention;
[0025] Figure 2 This is a schematic diagram of the process of performing posture analysis in combination with sensor data in the first embodiment of the present invention;
[0026] Figure 3 This is a flow chart of the particle swarm optimization algorithm in Example 1 of the present invention. DETAILED DESCRIPTION
[0027] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.
[0028] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.
[0029] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.
[0030] Embodiment 1
[0031] This embodiment discloses a human-computer interaction control method for a wearable flexible external limb, comprising:
[0032] By means of inertial sensors arranged at the joints of the flexible external limb, the measurement values of the external limb when performing the task are obtained;
[0033] Define the state vector of the Kalman filter, and obtain the state transfer matrix based on the current measurement value of the inertial sensor and the predicted value at the next moment;
[0034] Establishing the observation equation of Kalman filter, and obtaining the state covariance matrix predicted by Kalman filter based on the state transfer matrix;
[0035] Based on the observation values of the inertial sensors and Hall sensors set at the joints of the flexible external limbs, the state covariance matrix and the state vector are updated to obtain the joint pose of the flexible external limbs;
[0036] The obtained joint posture of the flexible external limb is used as feedback data and combined with the target motion data obtained based on the symmetry of the human kinematic chain to adjust the posture of the flexible external limb.
[0037] Combine the following Figure 1 A human-computer interaction control method for a wearable flexible external limb proposed in this embodiment is described in detail, specifically including:
[0038] S1: Place sensors at the joints of the wearer's limbs where they need to perform tasks.
[0039] S1 includes the following steps:
[0040] S1.1: Install IMU and Hall sensors at the wearer’s joints.
[0041] The IMU is used to obtain the acceleration, angular velocity and magnetic field direction data of the flexible external limbs, while the Hall sensor is used to obtain the absolute angle data of the flexible external limbs.
[0042] S2: Capture limb motion data in real time and process the data to ensure data stability and accuracy.
[0043] like Figure 2 As shown, S2 includes the following steps:
[0044] S2.1: Perform initial calibration on the IMU to eliminate bias error and scaling error. Calibrate the Hall sensor in a known magnetic field to determine the relationship between the magnetic field strength and direction and the sensor output.
[0045] S2.2: Perform preliminary pose estimation, estimate the pitch angle, roll angle and azimuth angle, and calculate the angular change of the pose by numerical integration.
[0046] S2.3: Read sensor data in real time and use Kalman filter for more accurate data fusion to provide more accurate pose estimation.
[0047] In the pose estimation of the flexible external limb, we first define a state vector X k , including information such as joint angle, angular velocity and acceleration.
[0048] The state transition equation describes the dynamic changes of the system. The angular velocity measurement based on the IMU can get the change of angle. By integrating the angular velocity, the angle at the next moment is obtained:
[0049]
[0050] Where: X k-1 It is the state estimation obtained by the Kalman filter result at the previous moment, combining the data of IMU and Hall sensor; is the predicted state vector; A is the state transfer matrix, which is used to predict the next state of the system; B is the control input matrix, Uk is the control input, which is set to 0 in this application; W k is the process noise, representing the uncertainty in the state model.
[0051] The observation equation is used to describe the relationship between the measurement value obtained from the sensor and the system state, which can be written as:
[0052] Z k =HX k +V k
[0053] Where: Z k is the observation data combined with the IMU and Hall sensor; H is the observation matrix, mapping the state vector to the measurement value; V k is the measurement noise, which represents the measurement error of the sensor.
[0054] Kalman filtering is divided into two main steps: prediction and update.
[0055] Use the state transition equation to predict the current state based on the previous state and sensor data:
[0056]
[0057] in, is the predicted state covariance, and Q is the process noise covariance matrix, which represents the uncertainty of the prediction.
[0058] The update step combines the actual measurements from the Hall sensors and IMU to correct the predicted state and prevent drift during long runs.
[0059] Kalman gain K k The weight used to weigh the predicted value against the measured value.
[0060]
[0061] Where R is the measurement noise covariance matrix, which represents the uncertainty of the sensor measurement.
[0062] Combine the predicted state with the actual measured data to update the current state estimate:
[0063]
[0064] And update the covariance:
[0065]
[0066] S3: Based on the principle of symmetry, the wearer's motion data is mapped into symmetrical motion data of the flexible external limbs.
[0067] like Figure 3As shown, S3 includes the following steps:
[0068] S3.1: Based on the symmetry of the human kinematic chain, the wearer's midline is used as the symmetry axis, usually an axis running vertically downward from the top of the head to the pelvis, which divides the body into left and right parts. This axis will be used to define the mapping relationship between the joint movements on the left and right sides to ensure that the generated symmetrical movements conform to the natural symmetrical movement characteristics of the human body.
[0069] S3.2: The wearer's joint data is mirrored based on the symmetry axis, that is, the data of the left joint is mapped to the right joint, and vice versa. In this way, the complete symmetrical motion data is generated.
[0070] S3.3: Select an appropriate optimization algorithm to optimize the motion of the mapped flexible exolimb, for example, by reducing the sudden changes in joint angles, limiting the sudden changes in angular velocity, etc., to make it smoother and more natural, and avoid uncoordinated or stiff motions.
[0071] Take particle swarm optimization algorithm as an example:
[0072] Initialization: Each particle represents a combination of action data. The initial position of the particle is determined by mapping the generated symmetric motion data, and the speed of the particle indicates how fast the action changes.
[0073] Define the fitness function: measure smoothness and naturalness by calculating the change in joint angle, angular velocity, and acceleration between consecutive time steps. At time t and t+1, the mapped joint angle is θ t and θ t+1 , the mapped joint angular velocity is ω t and ω t+1 , the mapped joint acceleration is a t and a t+1 ,but:
[0074]
[0075] The above parts are weighted and combined into a comprehensive fitness function:
[0076] f=w1f angle +w2f v +w3f a
[0077] Among them, w1, w2, w3 are different weight coefficients.
[0078] The fitness function is used to evaluate the quality of the solution. The better the fitness value, the closer the solution represented by the particle is to the optimal solution, and it guides the search direction and whether a good enough solution has been found.
[0079] Update particle velocity:
[0080]
[0081] in: is the velocity of particle i in the kth generation; w is the inertia weight, which controls the exploration ability of the particle; c1 and c2 are learning factors used to balance individual experience and group experience; r1 and r2 are random numbers to ensure diversity; is the historical optimal position of particle i, g best is the global optimal position in the population, calculated by the fitness function.
[0082] If the current fitness value of the particle is better than the historical optimal value, the historical optimal position will be updated; update the particle position:
[0083]
[0084] Repeat fitness assessment and keep updating and g best If the fitness function converges or reaches the preset number of iterations, the global optimal solution g is obtained. best , generating the action data that the optimized flexible exolimb needs to perform.
[0085] S4: The mapped symmetrical motion data is transmitted to the control system of the flexible external limb, and the joint posture of the flexible external limb is detected in real time as feedback to drive the flexible external limb to perform corresponding symmetrical motions.
[0086] S4.1: Transmit the optimized symmetrical motion data to the control system of the flexible exolimb.
[0087] S4.2: Drive the flexible exolimb to the target position and posture and perform the corresponding symmetrical movements.
[0088] S4.3: Read the IMU and Hall sensor data installed in the flexible external limb joints in real time, and perform actual pose estimation according to the steps in S2.
[0089] S4.4: The actual posture estimation result after fusion is used as feedback data. By combining the feedback data with the symmetrical action data optimized by S3, the posture of the flexible external limb is continuously adjusted to achieve precise and symmetrical operation.
[0090] Embodiment 2
[0091] The purpose of this embodiment is to provide a wearable flexible external limb human-computer interaction control system, including:
[0092] An acquisition module, used for acquiring measurement values of the external limb when performing a task through inertial sensors arranged at the joints of the flexible external limb;
[0093] The calculation module is used to define the state vector of the Kalman filter and obtain the state transfer matrix based on the current measurement value of the inertial sensor and the predicted value at the next moment;
[0094] A prediction module, used to establish an observation equation of a Kalman filter, and obtain a state covariance matrix predicted by the Kalman filter based on the state transfer matrix;
[0095] An updating module is used to update the state covariance matrix and the state vector based on the observation values of the inertial sensor and the Hall sensor arranged at the joint of the flexible external limb to obtain the joint posture of the flexible external limb;
[0096] The adjustment module is used to use the obtained joint posture of the flexible external limb as feedback data, combined with the target action data obtained based on the symmetry of the human kinematic chain, to adjust the posture of the flexible external limb.
[0097] Embodiment 3
[0098] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.
[0099] Embodiment 4
[0100] The purpose of this embodiment is to provide a computer-readable storage medium.
[0101] A computer-readable storage medium stores a computer program, which executes the steps of the above method when executed by a processor.
[0102] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.
[0103] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.
[0104] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.
Claims
1. A human-computer interaction control method for a wearable flexible external limb, characterized in that: The method comprises: By means of inertial sensors arranged at the joints of the flexible external limb, the measurement values of the external limb when performing the task are obtained; Define the state vector of the Kalman filter, and obtain the state transfer matrix based on the current measurement value of the inertial sensor and the predicted value at the next moment; Establishing the observation equation of Kalman filter, and obtaining the state covariance matrix predicted by Kalman filter based on the state transfer matrix; Based on the observation values of the inertial sensors and Hall sensors set at the joints of the flexible external limbs, the state covariance matrix and the state vector are updated to obtain the joint pose of the flexible external limbs; The obtained joint posture of the flexible external limb is used as feedback data and combined with the target motion data obtained based on the symmetry of the human kinematic chain to adjust the posture of the flexible external limb.
2. A human-computer interaction control method for a wearable flexible external limb as claimed in claim 1, characterized in that: According to the angular velocity measurement value of the inertial sensor, the angle at the next moment is predicted based on the integrated angular velocity.
3. A human-computer interaction control method for a wearable flexible external limb as claimed in claim 1, characterized in that: The collected joint data are used to generate complete motion data of symmetrical limbs based on the symmetry of the kinematic chain. The generated complete motion data are optimized based on the particle swarm optimization algorithm to obtain the target motion data.
4. A human-computer interaction control method for a wearable flexible external limb as claimed in claim 3, characterized in that: The particle swarm optimization algorithm is used to optimize the generated complete motion data, specifically: Each particle represents a combination of action data. The initial position of the particle is determined by mapping the generated symmetric motion data, and the speed of the particle represents the speed of the action change. The fitness function is defined by the change in joint angle, angular velocity, and acceleration between consecutive time steps; Update particle velocity and position; Repeat the fitness evaluation and continuously update the historical optimal position of the particle and the global optimal position in the group until the fitness function converges or reaches the preset number of iterations to obtain the global optimal solution; According to the obtained global optimal solution, the target motion data of the optimized flexible external limb is generated.
5. A human-computer interaction control method for a wearable flexible external limb as claimed in claim 4, characterized in that: The fitness function is: f=w1f angle +w2f v +w3f a , Among them, at time t and t+1, the mapped joint angle is θ t and θ t+1 , the mapped joint angular velocity is ω t and ω t+1 , the mapped joint acceleration is a t and a t+1 , w1, w2, w3 are different weight coefficients.
6. A wearable flexible external limb human-computer interaction control system, characterized in that: The system comprises: An acquisition module, used for acquiring measurement values of the external limb when performing a task through inertial sensors arranged at the joints of the flexible external limb; The calculation module is used to define the state vector of the Kalman filter and obtain the state transfer matrix based on the current measurement value of the inertial sensor and the predicted value at the next moment; A prediction module, used to establish an observation equation of a Kalman filter, and obtain a state covariance matrix predicted by the Kalman filter based on the state transfer matrix; An updating module is used to update the state covariance matrix and the state vector based on the observation values of the inertial sensor and the Hall sensor arranged at the joint of the flexible external limb to obtain the joint posture of the flexible external limb; The adjustment module is used to use the obtained joint posture of the flexible external limb as feedback data, combined with the target action data obtained based on the symmetry of the human kinematic chain, to adjust the posture of the flexible external limb.
7. A wearable flexible external limb human-computer interaction control system as claimed in claim 6, characterized in that: It also includes an optimization module, which is used to generate complete motion data of symmetrical limbs based on the symmetry of the kinematic chain from the collected joint data, and optimize the generated complete motion data based on an optimization algorithm to obtain target motion data.
8. A wearable flexible external limb human-computer interaction control system as claimed in claim 6, characterized in that: In the calculation module, the angle at the next moment is predicted based on the angular velocity measured by the inertial sensor and the integrated angular velocity.
9. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, a human-computer interaction control method for a wearable flexible external limb is performed as described in any one of claims 1 to 5.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the human-computer interaction control method for a wearable flexible external limb as claimed in any one of claims 1 to 5 is executed.
Citation Information
Cited By
High-resolution flexible wearable system for real-time capture of human joint motion postures
CN122767632A
