Uncoupling robot control system and method based on multi-source visual fusion

By using multi-source vision fusion and a feedforward-feedback collaborative control mechanism, and utilizing information from RGB, depth, infrared, and inertial measurement units, high-precision hook recognition and dynamic compensation are achieved in complex environments, improving the stability and success rate of the robot's hook removal.

CN121105033AActive Publication Date: 2025-12-12ANHUI HUADIAN SUZHOU POWER GENERATION
5 Cites 15 Cited by

Patent Information

Application Number
CN202511631109.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-11-10
Publication Date
2025-12-12
Estimated Expiration
2045-11-10

AI Technical Summary

Technical Problem

Existing robot control systems struggle to achieve high-precision hook-unhooking operations in complex environments where the hook target swings significantly. They are also affected by changes in lighting, occlusion, and inertial measurement noise, leading to unstable recognition and inaccurate positioning.

Method used

Employing a multi-source vision fusion and feedforward-feedback collaborative control mechanism, this system utilizes information from RGB, depth, infrared, and inertial measurement units. Through deep learning models and predictive control algorithms, it achieves high-precision identification and dynamic compensation of the hook. Combined with the robot's kinematic model, it performs registration and coordinate transformation to generate a feedforward control trajectory. Pixel deviations are calculated using optical flow for real-time correction.

Benefits of technology

This invention enables accurate hook identification and positioning under extreme interference, and calculates the three-dimensional coordinates of the hook, improving the capture success rate and stability in dynamic scenarios, and solving the problems of response lag and inaccurate positioning in traditional solutions.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The embodiment of the invention provides an unhooking robot control method based on multi-source visual fusion, which is applied to the technical field of robot control and comprises the following steps: acquiring an RGB image, a depth image, an infrared image and IMU data through a multi-source sensing system mounted at the tail end of a robot; carrying out feature fusion identification by adopting a double-branch neural network, and outputting the boundary contour of the lifting hook and the three-dimensional coordinates of the optimal grabbing point; the visual coordinates are unified to a robot base coordinate system through a registration correction mechanism; a Transform prediction model is constructed based on the visual and inertial signals, and future pose changes of the lifting hook are estimated; a feedforward control track is generated to counteract swing of the lifting hook, and track correction is carried out in combination with visual servo feedback; and a joint instruction is generated through path planning and inverse kinematics solution, and the mechanical arm is driven to complete precise unhooking operation. According to the method, the recognition precision, the anti-interference capability and the operation success rate of unhooking operation in complex illumination and dynamic environments are effectively improved.
Need to check novelty before this filing date? Find Prior Art