Fastening Component Recognition Using Coarse-to-Fine Heat Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing fastening component recognition methods struggle to accurately identify and disassemble fastening components in used home appliances due to variations in position, state, corrosion, and deformation, making automation difficult.
Innovation Solution
A method involving coarse and fine recognition using deep learning models, where a coarse recognition model identifies fastening components as heat maps, calculates barycentric positions, and clips images using fixed rectangles centered on tool holes, followed by fine recognition to confirm the components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If deep learning models are trained using circumscribed rectangles of visible portions of fastening components, then the model can handle varying visibility, but the recognition accuracy decreases when parts are corroded or deformed
Solution Approach 1:
The patent segments the fastening component recognition into two distinct stages: coarse recognition that handles visibility variations using circumscribed rectangles, and fine recognition that achieves high accuracy using fixed-size rectangles centered on tool holes. This segmentation allows each stage to optimize for its specific function without compromise.
Solution Approach 2:
The coarse recognition stage performs preliminary action by identifying candidate regions and calculating barycentric positions before the fine recognition stage. This preliminary positioning using fixed rectangles centered on tool holes prepares the data for high-accuracy recognition while maintaining adaptability to visibility variations.
2Adaptability or versatility
If manual disassembly methods are used, then flexibility to handle various fastening component variations is maintained, but automation cannot be achieved
Solution Approach 1:
The patent replaces manual mechanical inspection with an automated vision system that uses deep learning models. The system substitutes human intuition and experience with algorithmic processing of heat maps and barycentric position calculations, achieving both automation and adaptability to various fastening component variations.
Solution Approach 2:
The system changes parameters from manual visual inspection to automated image processing parameters including heat map thresholding, barycentric position calculation, and fixed rectangle dimensions. These parameter changes enable automation while maintaining the ability to handle variations in fastening components through the fine recognition stage.
3Ease of manufacture
If color-based detection is used for fastening components, then simple detection is possible, but accuracy decreases when multiple colors are present
Solution Approach 1:
The patent transitions from one-dimensional color-based detection to two-dimensional spatial analysis using heat maps and barycentric positions. By adding the spatial dimension and using fixed rectangles centered on tool holes, the system achieves accurate detection regardless of color variations or multiple colors present in the fastening components.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A coarse recognition is executed by outputting, as heat map (6), a position and a region of a fastening component from image data for automatic disassembly (5) of disassembly target object for recognition (4) based on coarse recognition deep learning model (12). The coarse recognition deep learning model is trained by learning fastening component (62) of learning data (56) of target object for learning data generation (51) based on fastening component data and learning a non-fastening component as non-fastening component data. The fastening component data is based on an image clipped by first fixed rectangle (66) centered on tool hole(62a) of the fastening component from the learning data. the non-fastening component data is based on an image clipped by a second fixed rectangle centered on an arbitrary point from the learning data. The second fixed rectangle is identical to the first fixed rectangle. A barycentric position for each region from the heat map of the fastening component is calculated. The fastening component from the image data for automatic disassembly is clipped by a third fixed rectangle centered on the barycentric position as fastening component image for fine recognition (7) A fine recognition is executed by determining whether or not the fastening component image for fine recognition is the fastening component of the disassembly target object for recognition.