Lug Nut Center Detection Using 2D and 3D Vision Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current robotic systems for automated wheel removal and replacement face challenges in accurately locating lug nuts due to variations in wheel configurations and potential damage concerns, leading to inefficiencies and safety hazards.
Innovation Solution
A system that uses machine learning to combine 2D and 3D image data to refine the center point of a lug nut, leveraging the general shape and topography of the lug nut to improve accuracy, by processing images to identify edges and measuring radius distances for precise center point determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional manual wheel removal methods are used, then operational flexibility is maintained, but efficiency is low and safety hazards exist
Solution Approach 1:
The patent replaces manual mechanical wheel removal operations with an automated robotic system equipped with computer vision and machine learning capabilities. The robotic system uses cameras to capture images, processes them through ML models to identify lug nut positions, and automatically performs removal operations, thereby eliminating safety hazards to personnel while dramatically improving operational efficiency.
2Productivity
If robotic systems are deployed to automate wheel removal, then efficiency and safety are improved, but accuracy in locating lug nuts decreases due to wheel configuration variability
Solution Approach 1:
The patent transitions from analyzing two-dimensional images to utilizing three-dimensional point cloud data for locating lug nuts. The 3D point cloud provides depth information and spatial context that enables the system to accurately identify lug nut positions across various wheel configurations, lug nut patterns, and viewing angles, thereby maintaining high measurement precision while enabling automated operation.
Solution Approach 2:
The system dynamically adjusts processing parameters and thresholds based on the specific wheel configuration detected in the point cloud data. By changing parameters such as distance thresholds, point density requirements, and geometric feature detection criteria, the system adapts to different wheel types and lug nut arrangements, maintaining accurate location identification across diverse configurations.
3Measurement precision
If guidance systems are made more accurate to handle various wheel configurations, then lug nut location precision improves, but system complexity increases
Solution Approach 1:
The patent extracts and isolates the critical geometric features of lug nuts from the complex wheel assembly point cloud data. By focusing specifically on identifying lug nut boundaries, edges, and center points rather than processing the entire wheel structure, the system achieves high measurement precision while keeping the processing algorithm relatively simple and computationally efficient.
4Ease of operation
If manual operations are used for wheel removal, then system simplicity is maintained, but time consumption increases
Solution Approach 1:
The robotic system performs wheel removal operations autonomously without requiring manual intervention. The system self-locates the wheel, identifies lug nut positions using computer vision, plans removal paths, and executes the entire operation automatically. This self-service capability dramatically reduces time consumption while maintaining ease of operation through centralized control.
Data Source
AI summary
Described is a system (and method) for locating a center point of a lug nut for an automated vehicle wheel removal system. To improve the accuracy of the center point, the system may perform machine learning inferences using two-dimensional (2D) and three-dimensional (3D) image data. The system may process a 2D image to infer an initial center point, and potentially improve the accuracy by leveraging a 3D image. More particularly, the system may process a 3D image to infer a location of one or more edges (or edge points) around the perimeter of the lug nut and measure a set of distances between the initial center point and the located set of edges. The system may then refine (or adjust) the center point based on such measurements.