AR Assembly Assistant with ML Step Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current product assembly processes face inefficiencies due to human error, which is costly and time-consuming to mitigate through manual inspection and specialized guides or jigs, and existing machine learning models struggle with limited training data and variations in lighting and texture in industrial settings.
Innovation Solution
A system utilizing a machine learning model trained with augmented datasets combining two-dimensional images and three-dimensional digital models, employing deep learning and classic shape matching techniques for robust step recognition, and an augmented reality assistant to provide real-time guidance on component placement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If manual inspection and specialized guides or jigs are used to mitigate human error, then assembly accuracy is improved, but operating costs and time consumption increase
Solution Approach 1:
The patent replaces mechanical inspection systems and physical guides/jigs with a vision-based machine learning system. Image capture devices replace manual inspectors, and ML algorithms replace specialized guides, eliminating the need for physical intervention while maintaining or improving assembly accuracy.
Solution Approach 2:
The system creates digital copies (images) of the assembly process and analyzes them through machine learning models. Instead of physically inspecting each component, the system captures visual data and processes it algorithmically, significantly reducing time and cost while maintaining precision.
2Loss of time
If machine learning models are trained with limited data in industrial settings, then training time and data collection costs are reduced, but recognition accuracy under lighting and texture variations deteriorates
Solution Approach 1:
The patent transforms the training data by converting images into frequency domain representations using Fourier transforms. This parameter transformation allows the model to learn rotation-invariant features, improving recognition accuracy across different orientations and lighting conditions without requiring extensive varied training data.
Solution Approach 2:
The system moves from spatial domain image analysis to frequency domain analysis. By transforming images into the frequency domain, the model gains rotational invariance and can recognize components regardless of their orientation or lighting variations, effectively adding a new dimension of robustness to the recognition system.
Data Source
AI summary
A method and system for an augmented reality assistant that recognizes a step in a product assembly process and assists in the installation of a constituent component into a base component. That system having a prepopulated database of templates, the templates being generated based off of two-dimensional images and the related three-dimensional models. The template database is used to train a first machine learning model, that model configured to identify the step in the product assembly process based on an image captured from an image capture device. Verifying that determination by a second machine learning model. Presenting an AR assistant to the user to assist with that step based on the related template.