Robotic Vision Tracking With Heatmap Feedback for FTA Assembly
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current robotic systems in the final trim and assembly (FTA) stage of automotive assembly face challenges due to movement irregularities, vibrations, and balancing issues, which hinder accurate tracking and automation of FTA tasks.
Innovation Solution
A system that updates the training of a neural network using heatmaps generated from regression output, incorporating translation and error feedback to improve the accuracy and robustness of robotic motion control in FTA operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional three-dimensional model-based computer vision matching algorithms are used, then the system can perform basic tracking functions, but the tracking accuracy deteriorates due to movement irregularities, vibrations, and varying lighting conditions
Solution Approach 1:
The patent transforms the rigid 3D model-based matching approach into a flexible heatmap-based regression approach. By changing from discrete model parameters to continuous heatmap probability distributions, the system can adapt to movement irregularities and vibrations while maintaining tracking accuracy under varying lighting conditions
Solution Approach 2:
The patent implements feedback mechanisms where the neural network continuously refines tracking based on predicted position errors and heatmap confidence scores. The system uses feedback from previous frame predictions to adjust current frame tracking, improving both accuracy and stability in dynamic conditions
2Productivity
If robot motion control is implemented in FTA operations, then automation efficiency is improved, but the system becomes adversely affected by movement irregularities and vibrations during vehicle transport
Solution Approach 1:
The patent applies beforehand cushioning by training the neural network with diverse training data that includes various movement irregularities and vibration patterns. This pre-training prepares the system to compensate for such disturbances before they occur during actual FTA operations, maintaining motion control stability while achieving high automation efficiency
3Adaptability or versatility
If neural network training is performed with standard methods, then the model can learn basic patterns, but the model fails to generalize well to varying lighting conditions and parts color changes
Solution Approach 1:
The patent applies preliminary action by generating heatmaps for all training images before neural network training begins. These pre-computed heatmaps serve as ground truth labels that guide the network to learn position prediction from visual features, enabling the model to generalize to varying lighting and color conditions while maintaining high tracking accuracy
Data Source
AI summary
A robotic system for use in installing final trim and assembly part includes an auto-labeling system that combines images of a primary component, such as a vehicle, with those of computer based model, where feature based object tracking methods are used to compare the two. In some forms a camera can be mounted to a moveable robot, while in other the camera can be fixed in position relative to the robot. An artificial marker can be used in some forms. Robot movement tracking can also be used. A runtime operation can utilize a deep learning network to augment feature-based object tracking to aid in initializing a pose of the vehicle as well as an aid in restoring tracking if lost.