Robotic Arm Control From 2D Human Pose Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing humanoid robot control technologies rely on expensive and bulky 3D stereo cameras to capture human motion, which are not conducive for installation and use on humanoid robots, limiting their motion control capabilities.
Innovation Solution
A robot control method using a computer-implemented process that obtains human pose images with ordinary cameras, processes pixel information through lightweight open pose detection and convolutional neural networks to determine three-dimensional positional information of key points, and utilizes kinematics models to control robotic arms with reduced hardware requirements, eliminating the need for 3D stereo cameras.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D stereo cameras are used to capture human motion, then motion capture accuracy is improved, but device cost and size increase
Solution Approach 1:
The patent uses 2D camera images as a simplified copy or representation of the 3D scene, then reconstructs three-dimensional positional information through image processing algorithms. Instead of directly capturing 3D data with complex stereo cameras, the system captures 2D images and computationally derives depth and position information, thereby reducing hardware complexity while maintaining measurement accuracy
Solution Approach 2:
The patent replaces the mechanical/optical 3D stereo camera system with a computational image processing system. By substituting complex hardware-based 3D capture with 2D image acquisition followed by algorithmic processing (including convolutional neural networks and kinematics models), the system achieves 3D motion capture accuracy without the complexity of stereo camera hardware
2Measurement precision
If 3D stereo cameras are used to capture human motion, then motion capture accuracy is improved, but installation ease deteriorates
Solution Approach 1:
The system uses readily available 2D camera images as a substitute for specialized 3D stereo camera setups. Since 2D cameras are ubiquitous and easy to install, this approach dramatically improves installation ease while the image processing pipeline reconstructs the necessary 3D motion information to maintain capture accuracy
Solution Approach 2:
The patent makes the camera system universal by using standard 2D cameras that can be easily installed and are widely available, rather than specialized 3D stereo cameras. The multi-functional image processing system then handles both 2D pose detection and 3D position reconstruction, making the overall system more adaptable and easier to deploy
3Device complexity
If ordinary cameras with image processing are used, then device cost and size are reduced, but three-dimensional position detection capability must be achieved through processing
Solution Approach 1:
The patent transforms 2D image data into 3D positional information through computational processing. By adding the dimension of computational reconstruction, the system derives depth and three-dimensional coordinates from two-dimensional camera images, effectively converting a 2D detection problem into a 3D solution through mathematical and neural network processing
4Productivity
If lightweight pose detection and neural network processing are used, then processing efficiency is improved, but computational complexity increases
Solution Approach 1:
The system performs preliminary pose detection and key point identification using lightweight algorithms before applying more complex convolutional neural networks for 3D reconstruction. This staged approach processes data in sequence, performing simpler operations first to reduce the complexity and computational load of subsequent processing steps
Data Source
AI summary
The present disclosure discloses a robot control method as well as an apparatus, and a robot using the same. The method includes: obtaining a human pose image; obtaining pixel information of key points in the human pose image; obtaining three-dimensional positional information of key points of a human arm according to the pixel information of the preset key points; obtaining a robotic arm kinematics model of a robot; obtaining an angle of each joint in the robotic arm kinematics model according to the three-dimensional positional information of the key points of the human arm and the robotic arm kinematics model; and controlling an arm of the robot to perform a corresponding action according to the angle of each joint. The control method does not require a three-dimensional stereo camera to collect three-dimensional coordinates of a human body, which reduces the cost to a certain extent.