AI Assembly Monitoring Using BOM-Guided Image Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision systems for assembly monitoring lack adaptability to complex and varied manufacturing processes, requiring extensive manual intervention and struggling with changes in product designs or component configurations.
Innovation Solution
A computer-implemented method using artificial intelligence-based models for image encoding, instance segmentation, and bill of materials comparison, enabling dynamic adaptation to product variations through promptable segmentation and integration of industrial foundation models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If classical computer vision systems with hand-crafted functions and rules are used, then the system is easier to implement for clearly defined scenarios, but it lacks adaptability to changes or variations in complex assembly processes
Solution Approach 1:
The patent replaces classical mechanical rule-based image processing systems with an AI-based neural network system. The neural network learns assembly patterns and product variations automatically from training data, eliminating the need for manual rule configuration and enabling adaptive monitoring of complex assembly processes with varying product designs
Solution Approach 2:
The system changes from fixed rule parameters to dynamic learning parameters. The neural network adjusts its internal parameters (weights and biases) based on training data, allowing the system to adapt to different product variations and assembly scenarios without manual reconfiguration, while maintaining ease of deployment through automated learning
2Reliability
If rule-based systems with predefined rules are used, then decision-making is guided systematically, but adapting these rules to changes is time-consuming and challenging
Solution Approach 1:
The neural network system performs self-learning and self-adaptation through automated training on new data. When product variations or assembly process changes occur, the system can retrain on new examples without requiring manual rule updates, significantly reducing adaptation time while maintaining reliable systematic decision-making through consistent neural network inference
Solution Approach 2:
The system performs preliminary learning during a training phase using historical assembly data and product variations. This preliminary action prepares the neural network to handle future changes more efficiently, so when actual adaptations are needed, the system already has learned patterns that can be fine-tuned rather than built from scratch
3Productivity
If template matching is used, then comparison with predefined templates is efficient, but the system lacks robustness when faced with variations or deviations from expected templates
Solution Approach 1:
The system transitions from static template matching to dynamic neural network-based pattern recognition. The neural network can adapt its feature extraction and comparison processes based on the specific input, allowing efficient comparison while maintaining robustness to variations through learned invariances and ability to handle deviations from any single template
4Measurement precision
If manual calibration is performed, then the system can account for changes in lighting conditions and camera angles, but this reactive approach does not provide real-time adaptation to dynamic assembly processes
Solution Approach 1:
The patent replaces manual calibration procedures with automated neural network-based adaptation. The system learns to be invariant to lighting conditions, camera angles, and other environmental factors during training, enabling real-time adaptation without requiring manual calibration interventions, thus maintaining measurement precision while achieving real-time speed
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method for monitoring a manufacturing process for a product, comprising the steps of: a) capturing (II) image data of the manufactured product with its components, b) providing (RET) a bill of materials containing the components of the manufactured product, c) encoding (I_ENC) the image data by applying an artificial intelligence-based model and providing the encoded image data as an image embedding, d) encoding (P_ENC) the bill of materials by applying a provided artificial intelligence-based model and providing the encoded bill of materials as an input prompt, e) decoding (M_DEC) the image encoding using the input prompt, f) segmenting (SML) the decoded image encodings and generating masks and tags for the components of the decoded image embeddings, g) converting (CML) the masks and tags for the components according to the bill of materials, h) checking (CHK).whether the converted masks and identifiers of the components match the components in the bill of materials, and output of the test result.