Coordinated handling method and system for automatically adjusting postures to adapt to human joint torques
By using a robot vision system and a deep neural network model to predict joint torque in real time, and combining this with a biomechanical model to optimize posture, the problem of neglecting human joint mechanics in existing technologies is solved, thus achieving human health protection during collaborative handling.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI JIAOTONG UNIV
- Filing Date
- 2025-07-09
- Publication Date
- 2026-07-31
AI Technical Summary
Existing collaborative handling technologies neglect human joint mechanics, resulting in excessive loads on joints, an inability to accurately estimate the torque of major human joints in real time, difficulty in optimizing handling postures, and impact on the health of the handlers.
By acquiring human posture data through a robot vision system, combining it with the weight of the object, using a deep neural network model to predict joint torque in real time, and calculating the optimal posture based on a biomechanical model, the robot is controlled to adjust its movements to maintain a low joint stress state.
It enables real-time and accurate estimation of joint torque and posture optimization, ensuring that handlers maintain a healthy posture and reducing the risk of joint stress during collaborative handling.
Smart Images
Figure CN122480925A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of automatic control technology, and more specifically, to a collaborative transport method and system that automatically adjusts posture in real time to adapt to human joint torque. Background Technology
[0002] In modern industry and logistics, collaborative handling technology, with its high efficiency and flexibility, has become a key means to improve operational efficiency. Currently, robot collaboration methods based on visual servoing and force sensing are widely used in collaborative handling scenarios. These methods capture environmental information through vision systems and combine them with force sensors to perceive contact forces, enabling robots and humans to work collaboratively, thus improving handling efficiency and safety to a certain extent.
[0003] However, existing collaborative handling technologies generally suffer from a significant drawback: they neglect real-time evaluation and motion optimization of human joint mechanics. During handling, the magnitude of the torque exerted on the human joints directly impacts the handler's health. Due to the lack of attention to human joint mechanics, existing methods cannot accurately estimate the torque of major joints in real time. This can lead to handlers adopting improper postures during operations, causing excessive loads on the joints. Over time, this can easily result in joint injuries, muscle strain, and other health problems.
[0004] Meanwhile, existing technologies lack scientific basis for posture evaluation and optimization, and cannot effectively screen out the handling postures that minimize joint load. Furthermore, collaborative robots struggle to adjust their movements according to dynamic changes in human posture, failing to provide effective guidance to the handler and further exacerbating the unreasonable stress on the handler's joints.
[0005] A patent search revealed invention patent application number 202010053107.8, which discloses a human-robot collaborative handling system and its hybrid visual-touch control method. This system acquires the interaction force information between the robotic arm of the collaborative robot and the object being handled, and uses admittance control to generate the desired position of the object. It also acquires image information of the object and uses visual servo control to generate the desired posture and height. Finally, it generates a tracking controller based on a radial basis function neural network to control the collaborative robot to track the desired position and posture, enabling the robot to assist the operator in handling the object. While this patent combines a depth camera and a six-dimensional force / torque sensor to achieve visual servo and admittance control of the robot's handling behavior, it lacks online estimation of human joint torques and posture optimization based on biomechanical models, thus failing to truly guarantee the healthy joint balance of the handler.
[0006] In summary, given the problems of the existing technologies, researching a collaborative transport method and system that automatically adjusts posture to adapt to human joint torques has become a critical task that urgently needs to be addressed. Summary of the Invention
[0007] To address the shortcomings of existing technologies, the purpose of this invention is to provide a collaborative transport method and system that automatically adjusts posture to adapt to human joint torques.
[0008] A cooperative transport method for automatically adjusting posture to adapt to human joint torques, according to the present invention, includes the following steps:
[0009] Step S1: Obtain the human posture data of the handler through the robot vision system and transmit the human posture data to the central control unit;
[0010] Step S2: Measure the weight of the object being transported and transmit the weight to the central control unit;
[0011] Step S3: Based on human posture data and object weight, obtain the real-time torque prediction value of the main motion joint through a pre-trained deep neural network model;
[0012] Step S4: Based on the real-time torque prediction value, the force distribution of the human body under different postures is simulated through a preset biomechanical model to calculate the optimal handling posture with the least joint pressure.
[0013] Step S5: Based on the optimal handling posture, control the robot to adjust the motion parameters in real time so that the handler can maintain the optimal handling posture.
[0014] Step S6: During the collaboration process, the posture deviation is detected in real time, and the low joint stress state is maintained through feedback mechanisms or equipment adjustments.
[0015] Preferably, in step S1, the robot vision system captures the body posture and physical characteristics of the person handling the task in real time, and obtains human posture data by using key point detection and 3D reconstruction.
[0016] Preferably, in step S2, the weight of the object being moved is measured using a force / torque sensor built into the robot gripper or a hand-held load sensor.
[0017] Preferably, the deep neural network model is trained offline based on publicly labeled datasets and a physical simulation platform, and undergoes online adaptive correction using real-time collected operational data.
[0018] Preferably, step S3 includes the following sub-steps:
[0019] Step S3.1: Based on the publicly labeled dataset and physical simulation platform, perform offline training on the deep neural network model;
[0020] Step S3.2: Real-time operation data is collected using force / torque sensors or hand-held load sensors, as well as the robot vision system, to perform online adaptive correction on the deep neural network model.
[0021] Preferably, in step S5, the robot is controlled to adjust the arm height, gripper extension distance and deflection angle in real time according to the optimal handling posture, so that the handler maintains the optimal handling posture.
[0022] Preferably, in step S6, if the posture is detected to deviate from the optimal handling posture, a visual or tactile cue is issued, or the robot position is automatically readjusted to maintain a low joint stress state.
[0023] Preferably, in step S6, visual or tactile cues are fed back to the handler in real time through the robot's interactive interface or wearable device.
[0024] Preferably, the robot vision system, force / torque sensor or hand-held load sensor, deep neural network model and biomechanical model are all integrated into the central control unit to realize real-time data processing and decision optimization.
[0025] The present invention also provides a cooperative transport system that automatically adjusts posture to adapt to human joint torques, comprising:
[0026] Module M1 acquires the human posture data of the handler through the robot vision system and transmits the human posture data to the central control unit;
[0027] Module M2 measures the weight of the object being transported and transmits the weight to the central control unit.
[0028] Module M3, based on human posture data and object weight, uses a pre-trained deep neural network model to obtain real-time torque predictions for the main motion joints.
[0029] Module M4, based on real-time torque prediction values, uses a pre-set biomechanical model to simulate the force distribution on the human body under different postures and calculates the optimal handling posture with the least joint pressure.
[0030] Module M5 controls the robot to adjust motion parameters in real time based on the optimal handling posture, so that the handler can maintain the optimal handling posture.
[0031] Module M6 detects posture deviations in real time during collaboration and maintains a low joint stress state through feedback mechanisms or equipment adjustments.
[0032] Compared with the prior art, the present invention has the following beneficial effects:
[0033] 1. This invention combines real-time collected posture information with object weight information into a deep learning algorithm model, solving the problem of the inability to accurately estimate the torque of major human joints in real time, and realizing the online output of joint torque data.
[0034] 2. This invention utilizes a pre-set biomechanical model to evaluate the joint torques output by the deep learning algorithm, thus overcoming the lack of scientific basis in traditional posture evaluation and optimization, and calculating and selecting the handling postures that minimize joint load.
[0035] 3. This invention formulates a robot motion control strategy based on the optimal posture evaluation results, which solves the problem that collaborative robots cannot dynamically adjust their movements according to human posture, and can guide the handler to maintain a healthy posture with low joint stress in real time. Attached Figure Description
[0036] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:
[0037] Figure 1 This is a flowchart of a collaborative transport method for automatically adjusting posture to adapt to human joint torque in an embodiment of the present invention. Detailed Implementation
[0038] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.
[0039] This invention combines real-time monitoring of the human body's 3D key point posture by a robot vision system, measurement of object weight by force / torque sensors of a robot gripper or hand, with online estimation of torques of major human joints by a deep learning model. Based on a pre-set biomechanical model, the system rapidly evaluates and dynamically optimizes the human handling posture. According to the optimization results, the system controls the arm height, extension distance, and deflection angle of the collaborative robot in real time, guiding the handler to maintain a healthy posture with minimal joint stress, effectively improving the level of joint health protection during handling.
[0040] Example 1:
[0041] Figure 1 This is a flowchart of a collaborative transport method for real-time automatic posture adjustment to adapt to human joint torque in an embodiment of the present invention.
[0042] like Figure 1As shown, this embodiment provides a collaborative transport method that automatically adjusts posture in real time to adapt to human joint torques, including the following steps:
[0043] Step S1: Obtain the human posture data of the handler through the robot vision system and transmit the human posture data to the central control unit.
[0044] Specifically, in step S1, the robot's vision system captures the body posture and physical characteristics of the person handling the task in real time, and uses key point detection and 3D reconstruction to obtain human posture data.
[0045] Step S2: Measure the weight of the object being transported and transmit the weight to the central control unit.
[0046] Specifically, in step S2, the weight of the object being moved is measured using the force / torque sensor built into the robot gripper or a hand-held load sensor.
[0047] Step S3: Based on human posture data and object weight, obtain the real-time torque prediction value of the main motion joint through a pre-trained deep neural network model.
[0048] The main moving joints include the shoulder, elbow, and wrist joints, which are the joints that bear the main load during transportation.
[0049] Specifically, the deep neural network model is trained offline based on publicly labeled datasets and a physical simulation platform, and then undergoes online adaptive correction using real-time collected operational data.
[0050] In this embodiment, step S3 includes the following sub-steps:
[0051] Step S3.1: Based on the publicly labeled dataset and physical simulation platform, the deep neural network model is trained offline so that it can accurately predict joint torques under various handling actions.
[0052] Step S3.2: Real-time operation data is collected using force / torque sensors or hand-held load sensors, as well as the robot vision system. The deep neural network model is then subjected to online adaptive correction to improve its prediction accuracy in different handling scenarios.
[0053] Step S4: Based on the real-time torque prediction value, the force distribution of the human body under different postures is simulated through a preset biomechanical model to calculate the optimal handling posture with the least joint pressure.
[0054] Specifically, the pre-built biomechanical model is obtained through training a deep learning network. During deep learning model training, motion capture devices are used to capture the postures of two subjects' arms, torso, and legs. Based on this, the lengths of the torso and limbs, and joint angles, are assessed, thus abstracting and building a human body model. Simultaneously, pressure-sensing gloves are placed on the subjects' hands, or distributed force measurement devices are placed on the ground. Combined with pre-acquired object weight information and subject weight information, the additional torque exerted by the object on each joint is obtained using the torque formula M = r × F, where r represents the lever arm and F represents the force. The human body model of the two subjects, their weight information, the measured force information, walking speed information, estimated joint force information, and the object's weight and position information are then used to train the model, resulting in an estimation model of the joint forces exerted by different people under different postures and on different weights.
[0055] Step S5: Based on the optimal handling posture, control the robot to adjust the motion parameters in real time so that the handler can maintain the optimal handling posture.
[0056] In this embodiment, the real-time adjustment process is completed by the robot's built-in control system. That is, the robot is input with the desired pose, and the built-in control system completes the adjustment.
[0057] Specifically, in step S5, the robot is controlled to adjust the arm height, gripper extension distance and deflection angle in real time according to the optimal handling posture, so that the handler can maintain the optimal handling posture.
[0058] Step S6: During the collaboration process, the posture deviation is detected in real time, and the low joint stress state is maintained through feedback mechanisms or equipment adjustments.
[0059] Specifically, in step S6, if the posture is detected to deviate from the optimal handling posture, a visual or tactile cue is issued, or the robot position is automatically readjusted to maintain a low joint stress state.
[0060] In this embodiment, visual or tactile cues are provided to the handler in real time through the robot's interactive interface or wearable devices.
[0061] Furthermore, the robot vision system, force / torque sensor or hand-held load sensor, deep neural network model and biomechanical model are all integrated into the central control unit to achieve real-time data processing and decision optimization.
[0062] Example 2:
[0063] The present invention also provides a collaborative transport system that automatically adjusts posture to adapt to human joint torque in real time. The collaborative transport system that automatically adjusts posture to adapt to human joint torque in real time can be implemented by executing the process steps of the collaborative transport method that automatically adjusts posture to adapt to human joint torque in real time. That is, those skilled in the art can understand the collaborative transport method that automatically adjusts posture to adapt to human joint torque in real time as a preferred embodiment of the collaborative transport system that automatically adjusts posture to adapt to human joint torque in real time.
[0064] Specifically, this collaborative transport system, which automatically adjusts posture in real time to adapt to human joint torques, includes:
[0065] Module M1 acquires the human posture data of the handler through the robot vision system and transmits the human posture data to the central control unit;
[0066] Module M2 measures the weight of the object being transported and transmits the weight to the central control unit.
[0067] Module M3, based on human posture data and object weight, uses a pre-trained deep neural network model to obtain real-time torque predictions for the main motion joints.
[0068] In this embodiment, module M3 includes the following sub-modules:
[0069] Module M3.1 is used for offline training of deep neural network models based on publicly labeled datasets and physical simulation platforms.
[0070] Module M3.2 uses force / torque sensors or hand-held load sensors, along with a robot vision system, to collect real-time operational data and perform online adaptive correction on the deep neural network model.
[0071] Module M4, based on real-time torque prediction values, uses a pre-set biomechanical model to simulate the force distribution on the human body under different postures and calculates the optimal handling posture with the least joint pressure.
[0072] Module M5 controls the robot to adjust motion parameters in real time based on the optimal handling posture, so that the handler can maintain the optimal handling posture.
[0073] Module M6 detects posture deviations in real time during collaboration and maintains a low joint stress state through feedback mechanisms or equipment adjustments.
[0074] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.
[0075] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.
Claims
1. A collaborative transport method that automatically adjusts posture to adapt to human joint torques, characterized in that, Includes the following steps: Step S1: Obtain the human posture data of the handler through the robot vision system, and transmit the human posture data to the central control unit; Step S2: Measure the weight of the object being transported and transmit the weight to the central control unit; Step S3: Based on the human posture data and the weight of the object, obtain the real-time torque prediction value of the main motion joint through a pre-trained deep neural network model; Step S4: Based on the real-time torque prediction value, the force distribution of the human body under different postures is simulated through a preset biomechanical model to calculate the optimal handling posture with the least joint pressure. Step S5: Based on the optimal handling posture, control the robot to adjust the motion parameters in real time so that the handler maintains the optimal handling posture; Step S6: During the collaboration process, the posture deviation is detected in real time, and the low joint stress state is maintained through feedback mechanisms or equipment adjustments.
2. The cooperative transport method for automatically adjusting posture to adapt to human joint torques according to claim 1, characterized in that, In step S1, the robot vision system captures the body posture and physical characteristics of the person carrying the load in real time, and obtains human posture data by using key point detection and 3D reconstruction.
3. The cooperative transport method for automatically adjusting posture to adapt to human joint torques according to claim 1, characterized in that, In step S2, the weight of the object being moved is measured using a force / torque sensor built into the robot gripper or a hand-held load sensor.
4. The cooperative transport method for automatically adjusting posture to adapt to human joint torques according to claim 1, characterized in that, In step S3, the deep neural network model is trained offline based on a publicly labeled dataset and a physical simulation platform, and then undergoes online adaptive correction using real-time collected operational data.
5. The cooperative transport method for automatically adjusting posture to adapt to human joint torques according to claim 4, characterized in that, Step S3 includes the following sub-steps: Step S3.1: Based on the publicly labeled dataset and physical simulation platform, the deep neural network model is trained offline; Step S3.2: Real-time operation data is collected using the force / torque sensor or hand-held load sensor and the robot vision system to perform online adaptive correction on the deep neural network model.
6. The cooperative transport method for automatically adjusting posture to adapt to human joint torques according to claim 1, characterized in that, In step S5, based on the optimal handling posture, the robot is controlled to adjust the arm height, gripper extension distance and deflection angle in real time so that the handler maintains the optimal handling posture.
7. The cooperative transport method for automatically adjusting posture to adapt to human joint torques according to claim 1, characterized in that, In step S6, if a deviation from the optimal handling posture is detected, a visual or tactile cue is issued, or the robot position is automatically readjusted to maintain a low joint stress state.
8. The cooperative transport method for automatically adjusting posture to adapt to human joint torques according to claim 7, characterized in that, In step S6, the visual or tactile cues are fed back to the handler in real time through the robot's interactive interface or wearable device.
9. The cooperative transport method for automatically adjusting posture to adapt to human joint torques according to claim 5, characterized in that, The robot vision system, the force / torque sensor or hand-held load sensor, the deep neural network model, and the biomechanical model are all integrated into the central control unit to achieve real-time data processing and decision optimization.
10. A collaborative transport system that automatically adjusts posture to adapt to human joint torques, characterized in that, include: Module M1 acquires the human posture data of the handler through the robot vision system and transmits the human posture data to the central control unit; Module M2 measures the weight of the object being transported and transmits the weight to the central control unit; Module M3, based on the human posture data and the weight of the object, obtains the real-time torque prediction value of the main motion joint through a pre-trained deep neural network model; Module M4, based on the real-time torque prediction value, simulates the force distribution of the human body under different postures through a preset biomechanical model, and calculates the optimal handling posture with the least joint pressure. Module M5 controls the robot to adjust motion parameters in real time according to the optimal handling posture, so that the handler maintains the optimal handling posture; Module M6 detects posture deviations in real time during collaboration and maintains a low joint stress state through feedback mechanisms or equipment adjustments.