Large-scene physical man-machine cooperation operation force oscillation compensation strategy
By integrating multimodal sensors and human body simulation models, the operation force oscillation model is established, and the problem of operation force oscillation in the cooperation of physical human-machine in large scenes is solved, the operation stability and safety is improved, adapted to individual differences, and provided comfort and Efficient operating environment.
Patent Information
- Application Number
- CN202411112261.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-19
AI Technical Summary
In the cooperation of physical human-machine in large scenes, the operator's walking or other movements cause the hand operation force to oscillate, affecting the stability and accuracy of the operation. The prior art relying on single-modal sensors is difficult to fully capture the operator's movement state, and it is not Can fully consider individual differences, resulting in insufficient prediction and compensation.
By integrating the six-dimensional force sensor for the hand, the tactile pressure sensor for the foot and the inertia measurement unit, a multi-modal information perception platform is built, a motion database is built with the human body simulation model, an operating force oscillation model is established, and an operating force oscillation model is predicted and compensated for operating force oscillation in real time.
A comprehensive understanding of the operator's overall motion state is achieved, the stability and accuracy of the operation is improved, the operation risks are reduced, and the naturalness and safety of the operation are enhanced.
Smart Images

Figure CN118981898A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of human-machine interaction and robot control technology, and in particular to a large-scale physical human-machine collaborative operation force oscillation compensation strategy. Background Art
[0002] With the advancement of technology, physical human-machine collaboration systems have been widely used in industries such as industry, medical care, and service industries. In large-scale tasks, physical human-machine collaboration can combine the respective advantages of operators and robots, significantly improving production efficiency and operational safety. However, in large-scale physical human-machine collaboration, the operator's walking or other movements often cause oscillations in the hand's operating force, which affects the stability and accuracy of the operation. This problem is particularly prominent in large-scale collaborative tasks, because the operator's range of motion and the complexity of the operation increase, making the problem of operating force oscillation more serious.
[0003] In the prior art, the perception of the operator's intention usually relies on a single-modal sensor. For example, only a hand force sensor is used to measure the human body's intention. The limitation of this method is that it cannot fully capture the movement state of the operator's entire body. Due to the lack of comprehensive motion data, it is difficult for a single-modal sensor to accurately predict and compensate for the operation force oscillation, which will cause instability and operation errors in physical human-machine collaboration. In addition, the prediction models in the prior art often fail to fully consider the individual differences of the operator, such as height and weight, which limits the accuracy and applicability of the prediction model. Therefore, in order to solve the problem of operation force oscillation caused by the operator's walking in large-scale physical human-machine collaboration and improve the stability of the operation process, it is necessary to develop a physical human-machine collaboration operation force oscillation compensation strategy. This strategy should be able to adapt to the individual differences of the operator to provide a smooth operation experience, thereby providing a theoretical basis for the operator's smooth operation. In this way, the overall performance and operational safety of the human-machine collaboration system can be significantly improved. Summary of the invention
[0004] In view of the shortcomings of the prior art, the present invention provides a large-scale physical human-machine collaborative operation force oscillation compensation strategy, which can accurately predict and compensate for the force oscillation during the operation in real time, improve the stability and accuracy of the operation, and reduce the operator's operation risk. By integrating the sensor data of the hands and feet, and building a human motion database in combination with a human body simulation model, and establishing an operation force oscillation model based on a mapping relationship between the hand and foot motion information, the present invention can more comprehensively understand the operator's overall motion state, and accordingly design an effective compensation strategy, thereby providing the operator with a more comfortable and efficient working environment.
[0005] The technical solution adopted by the present invention to solve its technical problems is: a large-scale physical human-machine collaborative operation force oscillation compensation strategy, including the following steps: step one, considering the motion coordination characteristics of the operator's single-handed operation, and building a multimodal information perception platform; step two, using actual data and human body model simulation data to analyze the operator's movement, and build a human body movement database; step three, combining the mapping relationship between foot movement data and hand operation force data to predict the operation force oscillation node and amplitude, and establish the operator's operation force oscillation model; step four, real-time compensation for the operation force oscillation in the physical human-machine collaboration process.
[0006] Furthermore, step one includes: determining the motion coordination characteristics of the operator when operating with one hand, in which the torso is toward or away from the force application point, and the friction of the sole affects the size and direction of the hand force; selecting sensors that can sense key parameters of human motion, including a six-dimensional force sensor for the hand, a tactile pressure sensor for the foot, and an inertial measurement unit; the six-dimensional force sensor for the hand is fixed at the end of the robotic arm, and can measure the hand operation information in real time, including the size, direction and point of action of the operation force; the tactile pressure sensor for the foot is fixed in the operator's shoe, and can measure the foot tactile pressure distribution data and pressure change rate in real time, reflecting the contact between the sole and the ground during the operator's walking; the inertial measurement unit is fixed at the toe position, and can measure the operator's walking angular velocity and acceleration, as well as the relative position of the toe and the ground in real time; designing the layout and installation method of the sensor to ensure the synchronization and consistency of the sensor data, and developing a data acquisition system for realizing real-time acquisition and transmission of sensor data, and transmitting the data to the central processing unit through a corresponding communication method.
[0007] Furthermore, step two includes: synchronously collecting hand six-dimensional force sensors, foot tactile pressure sensors, inertial measurement units and motion capture data, and ensuring data time synchronization; building a human body model, and inputting the human motion data captured by the motion capture system into the Anybody human biomechanical model for kinematic simulation; combining actual data with simulation data to analyze the operator's motion characteristics, building a human motion database, and storing the operator's motion characteristic data for subsequent prediction and compensation analysis.
[0008] Furthermore, step three includes: finding the mapping rules between foot motion data and hand operation force data, and establishing a mapping relationship; using the established mapping relationship to perform time series analysis, and predicting the nodes and amplitude of the operation force oscillation through foot motion data combined with a machine learning algorithm; combining the data in the human motion database to establish a mathematical model that can describe the operation force oscillation characteristics, and adjust it according to the height and weight differences of the operator.
[0009] Furthermore, step four includes: designing a compensation strategy based on the operator's motion characteristics and the operating force oscillation model, which strategy can be optimized according to the prediction model and actual experimental data to improve the compensation effect; real-time collection of hand six-dimensional force sensor and foot sensor data, and using the prediction model to analyze the real-time collected data to predict the nodes and amplitude of the operating force oscillation; based on the prediction results, real-time adjustment of the robot control parameters to achieve real-time compensation for operating fluctuations.
[0010] Compared with the prior art, the present invention has the following beneficial effects:
[0011] The intention perception platform in the prior art is often limited to single-modal sensors. For example, it only relies on hand force sensors to measure human intentions. This method cannot fully reflect the operator's overall motion state. The present invention integrates the data of six-dimensional hand force sensors and foot sensors, and builds a human motion database in combination with a human simulation model, thereby achieving a comprehensive understanding of the operator's overall motion state.
[0012] The present invention predicts the key nodes and amplitude of the operating force oscillation by real-time analysis of the hand six-dimensional force sensor and foot sensor data, combined with an accurate operating force oscillation model. At the same time, the present invention takes into account the individual differences of operators, such as height and weight, and adjusts the robot control parameters to achieve real-time compensation for operating fluctuations. This innovation not only improves the stability and efficiency of human-machine collaboration, but also significantly enhances the naturalness and safety of operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0014] Figure 1 It is a schematic diagram of the execution flow of a large-scale physical human-machine collaborative operation force oscillation compensation strategy provided by an embodiment of the present invention;
[0015] Figure 2 is a schematic diagram of the execution flow of step 1 provided in an embodiment of the present invention;
[0016] Figure 3 It is the multimodal information perception platform in step 1 provided in the embodiment of the present invention;
[0017] Figure 4 is a schematic diagram of the execution flow of step 2 provided in an embodiment of the present invention;
[0018] Figure 5is a schematic diagram of the execution flow of step three provided in an embodiment of the present invention;
[0019] Figure 6 It is a schematic diagram of the execution flow of step 4 provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0020] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
[0021] refer to Figure 1 , a large-scale physical human-machine collaborative operation force oscillation compensation strategy, including the following steps: step one, considering the motion coordination characteristics of the operator's single-handed operation, and building a multimodal information perception platform; step two, using actual data and human body model simulation data to analyze the operator's motion, and building a human body motion database; step three, combining the mapping relationship between foot motion data and hand operation force data to predict the operation force oscillation node and amplitude, and establish the operator's operation force oscillation model; step four, real-time compensation for the operation force oscillation in the physical human-machine collaboration process.
[0022] refer to Figure 2 , step one includes:
[0023] First, the motion coordination characteristics of the operator during one-handed operation are determined. The relationship between the operator's torso orientation and the force application point is observed and analyzed through video analysis, motion capture technology, and direct observation to determine the impact of torso movement on the operating force.
[0024] Secondly, according to the characteristics of motion coordination, sensors that can sense the key parameters of human motion are selected, including six-dimensional force sensors for the hands, tactile pressure sensors for the feet, and inertial measurement units, to ensure that the sensors have sufficient accuracy, response speed, and robustness to adapt to different operating environments;
[0025] Then, refer to Figure 3 , design the layout and installation of sensors to ensure the synchronization and consistency of sensor data:
[0026] The six-dimensional force sensor of the hand is fixed at the end of the robotic arm to ensure that the operation force information of the hand can be measured in real time, including the magnitude of the force (F x , F y , F z , T x , T y , T z ), direction (θ x ,θ y ,θ z ) and the point of action (P x , P y , P z );
[0027] The foot tactile pressure sensor is installed in the operator's shoe to measure the tactile pressure distribution (P f (i,j)) and pressure change rate (dP f (i,j) / dt), where i and j represent the row and column positions of the pressure sensor;
[0028] The inertial measurement unit is fixed at the toe of the shoe to measure the angular velocity (ω x ,ω y ,ω z ), acceleration (a x , a y , a z ), and the relative position of the toe of the shoe to the ground (Δx, Δy, Δz);
[0029] Finally, a data acquisition system is developed to realize real-time acquisition, preprocessing and transmission of sensor data. The data acquisition system includes a microcontroller, a signal conditioning circuit and a data transmission interface. Different communication methods are selected for different sensors to transmit data to the central processing unit for further processing and analysis. The six-dimensional force sensor of the hand uses Ethernet communication, and the tactile pressure sensor of the foot and the inertial measurement unit use Bluetooth communication. This ensures that the operator's motion coordination characteristics during one-handed operation are accurately perceived and recorded, providing a reliable data basis for subsequent prediction and compensation of operating force oscillations.
[0030] refer to Figure 4 , step 2 includes:
[0031] First, fix the six-dimensional force sensor of the hand, the tactile pressure sensor of the foot, and the inertial measurement unit at the designated position, and wear the motion capture suit required by the motion capture system, with the motion reflective markers affixed on it, and pull the robot within the motion range of the motion capture system to perform collaborative tasks; use timestamps to ensure that the data collection of the six-dimensional force sensor of the hand, the tactile pressure sensor of the foot, the inertial measurement unit, and the motion capture system are completely aligned in time; transmit the data of the sensor and the motion capture system to the data processing unit in real time through the data acquisition device for analysis and processing;
[0032] Secondly, a full-body human simulation model is selected, and the human motion data captured by the motion capture system is input into the Anybody human biomechanics model for kinematic simulation to ensure that it can accurately simulate the operator's motion process; the operator's motion characteristics are analyzed through the model, and the kinematic parameters of each joint and limb are simulated and output;
[0033] Finally, the operator’s motion characteristics are analyzed by combining the actual collected sensor data with the simulation data, and the two data are mutually verified to prove the effectiveness of data collection. A human motion database is built for subsequent prediction and compensation analysis.
[0034] refer to Figure 5 , step three includes:
[0035] First, establish the mapping relationship between foot motion data and hand operation force data:
[0036] Define the foot motion data feature vector F:
[0037] F=[F1,F2,F3,F4,F5] T ,
[0038] Where: F1 = [P f (i, j)] T ; F2 = [dP f(i,j) / dt] T ; F3 = [ω x ,ω y ,ω z ] T ; F4 = [a x , a y , a z ] T ; F5 = [Δx, Δy, Δz] T .
[0039] Define the hand operation force data feature vector H:
[0040] H=[H1,H2,H3] T ,
[0041] Where: H1=[F x , F y , F z , T x , T y , T z ] T ; H2=[θ x ,θ y ,θ z ] T ; H3=[P x , P y , P z ] T .
[0042] Using the linear regression model to establish the mapping relationship, it can be expressed as:
[0043] H=B·F+e
[0044] Where: B is the regression coefficient matrix; e is the error vector.
[0045] The regression coefficient B is calculated using the least squares method. The calculation formula is:
[0046] B=(F T F) -1 ·F T ·H
[0047] Among them: F T is the transpose of the eigenvector F.
[0048] Using the established mapping relationship, the input foot motion data is analyzed using the autoregressive integral moving average model to predict the nodes and amplitude of the operating force oscillation. Combined with the data from the human motion database, the prediction model for operators of different heights and weights is trained so that it can be adaptively adjusted based on individual differences.
[0049] refer to Figure 6 , step four includes:
[0050] First, the operator's height and weight parameters are input, and the data of the hand six-dimensional force sensor and foot sensor are collected in real time. The prediction model is used to analyze the real-time collected data to predict the node t and amplitude O(t) of the operating force oscillation;
[0051] Then, when reaching the oscillation node, the robot starts the operating force oscillation compensation strategy and adjusts the robot control parameters in real time based on the oscillation amplitude O(t);
[0052] The formula for admittance control is expressed as:
[0053]
[0054] Among them: F is the six-dimensional force; x is the displacement of the system; M is the inertia coefficient, which represents the mass or inertia of the system; B is the damping coefficient, which represents the damping characteristics of the system; K is the stiffness coefficient, which represents the stiffness of the system.
[0055] In order to adapt to the oscillation of the hand operation force, the dynamic response characteristics of the robot are improved by adjusting the parameters in the admittance matrix. The adjusted parameters B′ are as follows:
[0056] B′=B+ΔB
[0057] Where: B is the adjusted admittance control parameter; ΔB is the adjustment amount.
[0058] Finally, during the operation, the hand operation force oscillation is monitored in real time, and the admittance control parameters are adjusted according to the prediction model. The adjusted admittance control parameters are used to control the dynamic response of the robot to reduce the impact of the oscillation on the operation. The admittance control strategy is combined with the closed-loop control of the robot control system to achieve more stable operation.
[0059] The above is only a specific embodiment of the present invention, and cannot be used to limit the scope of the invention. Therefore, the replacement of equivalent components, or equivalent changes and modifications made according to the scope of protection of the patent of the present invention, should still fall within the scope of this patent. In addition, the technical features of the present invention can be freely combined with each other, with each other, and with each other to achieve the technical effects of the present invention.
Claims
1. A large-scale physical human-machine collaborative operation force oscillation compensation strategy, characterized in that The following steps are involved: (1) Step 1: Consider the motion coordination characteristics of the operator's one-handed operation and build a multimodal information perception platform, including: determining the motion coordination characteristics of the operator's one-handed operation, in which the torso is facing or away from the force application point, and the friction of the sole affects the magnitude and direction of the hand force; selecting sensors that can sense key parameters of human motion, including six-dimensional force sensors for the hand, tactile pressure sensors for the foot, and inertial measurement units; designing the layout and installation method of the sensors to ensure the synchronization and consistency of sensor data, and developing a data acquisition system to realize real-time acquisition and transmission of sensor data; (2) Step 2: Analyze the operator's motion using actual data and human body model simulation data, and build a human body motion database, including: synchronously collect hand six-dimensional force sensor, foot tactile pressure sensor, inertial measurement unit and motion capture data, and ensure data time synchronization; build a human body model, input the human body motion data captured by the motion capture system into the Anybody human biomechanical model for kinematic simulation; combine the actual data with the simulation data to analyze the operator's motion characteristics, build a human body motion database, and store the operator's motion characteristic data for subsequent prediction and compensation analysis; (3) Step 3: Combine the mapping relationship between the foot motion data and the hand operation force data to predict the operation force oscillation nodes and amplitude, and establish the operator's operation force oscillation model, including: finding the mapping law between the foot motion data and the hand operation force data, and establishing the mapping relationship; using the established mapping relationship to perform time series analysis, and predicting the operation force oscillation nodes and amplitude through the foot motion data combined with the machine learning algorithm; combining the data in the human motion database to establish a mathematical model that can describe the operation force oscillation characteristics, and adjust it according to the height and weight differences of the operator; (4) Step 4: Real-time compensation for the operating force oscillation in the physical human-machine collaboration process, including: designing a compensation strategy based on the operator's motion characteristics and the operating force oscillation model, which can be optimized according to the prediction model and actual experimental data to improve the compensation effect; real-time collection of hand six-dimensional force sensor and foot sensor data, and using the prediction model to analyze the real-time collected data to predict the nodes and amplitude of the operating force oscillation; based on the prediction results, real-time adjustment of the robot control parameters to achieve real-time compensation for operating fluctuations.
2. The compensation strategy according to claim 1, wherein: The multimodal information perception platform further includes: a six-dimensional hand force sensor fixed at the end of the robotic arm, which can measure the hand operation information in real time, including the size, direction and point of application of the operation force; a foot tactile pressure sensor fixed in the operator's shoe, which can measure the foot tactile pressure distribution data and pressure change rate in real time, reflecting the contact between the operator's sole and the ground during walking; an inertial measurement unit fixed at the toe position, which can measure the operator's walking angular velocity and acceleration, as well as the relative position of the toe and the ground in real time; a data acquisition system, which is used to realize real-time acquisition and transmission of sensor data, including data preprocessing, synchronization and compression, and transmit the data to the central processing unit through corresponding communication methods.
3. The compensation strategy according to claim 1, wherein: The human body simulation model is an Anybody human biomechanical model, which is used for kinematic simulation. The model can simulate the dynamic behavior of the operator during walking, including the movement trajectory of various parts of the body, joint force and muscle force, etc.
4. The compensation strategy according to claim 1, wherein: The operating force oscillation model is established through the mapping relationship between the hand and the foot and the machine learning algorithm, and is used to predict the operating force oscillation. The model can predict the occurrence time and amplitude of the operating force oscillation based on the foot movement data and the data stored in the human body database by modifying the operator's height and weight parameters.
5. The compensation strategy according to claim 1, wherein: The real-time compensation includes designing a compensation strategy, which analyzes the data collected by the hand six-dimensional force sensor and the foot sensor in real time, combines the operation force oscillation model to predict the nodes and amplitude of the operation force oscillation, and uses an adaptive control strategy to adjust the robot motion parameters accordingly to achieve real-time compensation for operation fluctuations.
