Robotic Vision Pose Estimation for Precise Brick Placement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vision systems for robotic construction equipment face challenges in accurately determining the location of objects, such as bricks, due to spatial constraints and varying outdoor conditions, which hinders precise placement.
Innovation Solution
A method involving capturing images of two faces of an object using stereoscopic cameras or Time of Flight sensors to generate point clouds, fitting planes, and determining the object's pose in a local coordinate system, allowing for precise positioning of the object for gripping and placement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a vision system is used to determine the exact 6DOF location of the brick, then the placement precision is improved, but the system complexity and processing time increase
Solution Approach 1:
The patent uses a simplified 2D shape model (copy) of the brick's top face instead of complex 3D modeling or full 6DOF vision systems. The shape model captures essential geometric features (edges, corners, dimensions) needed for localization while reducing computational complexity. This allows the system to achieve sub-mm placement precision without requiring full 6DOF vision system complexity.
Solution Approach 2:
The patent extracts only the necessary information from the brick - specifically the 2D shape and orientation of the top face - rather than attempting to capture or process all 6 degrees of freedom. By taking out only the essential geometric features needed for placement, the system achieves high precision while avoiding the complexity of complete 6DOF measurement.
2Measurement precision
If multiple cameras and sensors are used to capture images of the brick, then the measurement precision is improved, but the spatial constraints at the layhead are violated
Solution Approach 1:
The patent uses only one or two cameras positioned at the layhead to capture images of the brick's top face, rather than deploying multiple cameras from all angles. This partial action approach captures sufficient geometric information (2D shape and orientation) for precise placement while fitting within the limited spatial constraints of the layhead environment.
Solution Approach 2:
The patent transitions from attempting to measure all 6 degrees of freedom in 3D space to focusing on the 2D geometry of the brick's top face. By changing the dimensional approach from full 6DOF to 2D shape analysis, the system achieves sufficient measurement precision for placement while requiring minimal spatial resources at the constrained layhead location.
3Productivity
If the vision system processes images quickly to maintain rapid bricklaying speed, then the productivity is improved, but the measurement precision may be compromised
Solution Approach 1:
The patent extracts only the essential 2D shape features from the brick's top face image, ignoring unnecessary information. This extraction of minimal sufficient data reduces processing time and maintains high bricklaying speed while preserving the measurement precision needed for accurate placement.
Solution Approach 2:
The system performs partial processing by focusing only on the critical geometric features of the brick's top face rather than analyzing the entire object in full 6DOF. This partial action approach maintains processing speed for rapid bricklaying while achieving sufficient precision for accurate placement.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables accurate determination of an object's pose in 6DOF with high precision and speed, even in constrained environments and adverse conditions, facilitating precise placement by robotic gripper arms.
Implementation Method 1
capturing images of two faces of an object using stereoscopic cameras or Time of Flight sensors to generate point clouds
Data Source
AI summary
The present disclosure relates to a method for determining a pose of an object in a local coordinate system of a robotic machine, the method including: capturing at least one image of a first face of the object and at least one image of at least a portion of a second face of the object; generating a point cloud representation of at least part of the object using image data obtained from the captured images of the first and second faces; fitting a first plane to the first face of the object and fitting a second plane to the second face of the object using the point cloud representation; determining a pose of the first plane and a pose of the second plane; retrieving a shape model of at least the first face of the object; locating the shape model in the local coordinate system using at least in part the at least one image of the first face; and, determining the pose of the object in the local coordinate system. A vision system and robotic machine are also disclosed.