一种基于视觉语言动作模型的灵巧手运动规划与控制方法
By using a visual language motion model, the control bottleneck of dexterous hands in complex scenarios was solved, achieving efficient, stable and smooth motion control with limited data, and improving the generalization ability and reliability of dexterous hands in multimodal human-machine collaborative scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN RUIYAN INTELLIGENT CONTROL CO LTD
- Filing Date
- 2026-01-19
- Publication Date
- 2026-07-17
AI Technical Summary
Traditional dexterous hands face multiple bottlenecks in terms of data and performance, including high-cost data acquisition, inefficient reinforcement learning, difficulty in generating action sequences for fine control requirements with single-modal models, and motion jitter and error accumulation during the deployment phase, which limit their generalization ability and reliability in complex multimodal human-machine collaborative scenarios.
A visual-language action model-based approach is adopted. By synchronizing multi-channel camera images and language task commands in a timely manner, combined with dynamic adaptive exponential moving average filtering, feature fusion of visual encoder and language encoder, and a double buffering mechanism to train the model, a smooth control sequence is generated through time fusion, speed limiting and Bezier interpolation. The deployed thread monitors in real time and allows for manual intervention, forming a robust control system.
With limited teaching data, the practicality and engineering deployability of the dexterous hand were improved, high-frequency jitter was suppressed, grasping stability and real-time response capabilities were enhanced, robustness to changes in lighting and external disturbances was improved, data acquisition costs and manual annotation requirements were reduced, and operational safety and fault recovery capabilities were enhanced.
Smart Images
Figure CN121696972B_ABST