Automobile IBC product off-line appearance identification detection system and method thereof

Abstract

The application provides an automobile IBC product offline appearance identification detection system and method, and relates to the technical field of automobile part detection. The system comprises: a feeding vision module for identifying the placement posture of the IBC product to be detected on the conveying line and outputting posture data; a mechanical hand module for grabbing the product according to the posture data and transferring it to a detection station; a rotary table module arranged at the detection station for fixing and rotating the product; a vision detection module for scanning the surface of the product during rotation, acquiring three-dimensional image data, and performing defect detection and classification based on the three-dimensional image data; a data tracing module in communication with a central control system and an enterprise MES system for binding the unique identification of the product and the detection data; and the central control system for coordinating the operation of each module and controlling the mechanical hand module to perform a sorting and stacking operation corresponding to the result according to the result of defect detection and classification. Full automation and full coverage of the detection process are realized.

Description

Technical Field

[0001] This invention belongs to the field of automotive parts testing technology, specifically relating to an automotive IBC product off-line appearance recognition testing system and method. Background Technology

[0002] In the automotive manufacturing industry, critical safety components such as IBCs (Integrated Brake Controllers) must undergo rigorous visual and assembly condition inspections before leaving the production line. Traditional inspection methods primarily rely on manual visual inspection or random sampling using fixed industrial cameras, which have significant drawbacks. First, manual inspection is inefficient, labor-intensive, and susceptible to subjective factors, leading to inconsistent judgment standards and difficulty in ensuring quality stability. Second, for complex products like IBCs, which contain polyhedrals and micro-assembly features (such as bolts and pins), fixed-view cameras cannot achieve comprehensive, blind-spot-free scanning, posing a risk of missed inspections. Finally, inspection results are usually independent of the production information system, creating data silos. Defective products cannot be quickly traced back to the production batch or assembly station, hindering closed-loop management of quality issues and process optimization. Therefore, current technology lacks a solution that can automatically, comprehensively, and efficiently complete the visual inspection of complex automotive parts and deeply integrate with the production management system. Summary of the Invention

[0003] In view of the above-mentioned defects or deficiencies in the prior art, an automotive IBC product off-line appearance recognition inspection system and method are provided.

[0004] Firstly, this application proposes an automotive IBC product off-line appearance recognition inspection system, comprising:

[0005] The loading vision module is used to identify the placement posture of IBC products to be inspected on the conveyor line and output posture data;

[0006] The robotic arm module is used to grasp the product based on the posture data and transfer it to the inspection station;

[0007] A rotary table module, located at the testing station, is used to fix and rotate the product;

[0008] The visual inspection module is used to scan the surface of the product during rotation, acquire and perform defect detection and classification based on 3D image data;

[0009] The data traceability module communicates with the central control system and the enterprise MES system to bind the unique product identifier and test data.

[0010] The central control system is communicatively connected to the loading vision module, the robotic arm module, the rotary table module, the vision inspection module, and the data traceability module. The central control system is used to coordinate the operation of each module and, based on the results of defect detection and classification, control the robotic arm module to perform sorting and stacking operations corresponding to the results.

[0011] The central control system is configured for:

[0012] Based on the product model information, the corresponding 3D model and process data are called to divide the product surface into multiple functional areas, and each functional area is associated with its corresponding quality risk level.

[0013] The visual inspection module is controlled to construct a dual-channel heterogeneous processing architecture during the scanning process. The first channel is loaded with a lightweight convolutional neural network to extract features and classify defects in the macroscopic appearance surface area. The second channel is loaded with a graph neural network based on point cloud data to construct a topology map of the sub-region point cloud of the micro-assembly area and quantitatively analyze assembly state anomalies and micro-morphological deviations.

[0014] A cross-channel attention guidance mechanism is established. When the first channel detects a specific type of defect in the macroscopic appearance area, it sends a signal to the second channel to guide it to focus on analyzing the microscopic assembly area that is mechanically or technologically related to the macroscopic defect. Based on the results of defect detection and classification, the robot module is controlled to perform sorting and stacking operations corresponding to the results.

[0015] According to the technical solution provided in this application, the central control system is further configured to: trigger differentiated processing instructions linked with downstream production processes based on the results of defect detection and classification; wherein, the differentiated processing instructions include: generating and sending targeted warning information to downstream assembly processes for products with minor defects, and generating and sending isolation and quality anomaly notifications for products with serious defects.

[0016] According to the technical solution provided in this application, the central control system is also connected to a barcode scanning module; the central control system is used to instruct the barcode scanning module to read the product's unique identifier, and obtain the product model information from the MES system based on the unique identifier, and then call up the exclusive detection parameters and barcode placement rules that match the model.

[0017] According to the technical solution provided in this application, the visual inspection module includes a 3D laser line scanning camera and an AI image processing unit; the rotary table module is a high-precision servo rotary table; the central control system controls the rotary table module and the 3D laser line scanning camera to move synchronously, so as to realize 360° scanning of the product fixed on the rotary table module without blind spots; the AI ​​image processing unit identifies the three-dimensional image data obtained by scanning, including the determination of macroscopic appearance defects of the product and the microscopic details and assembly status of bolts, pins, oil ports, and assembly parts.

[0018] Secondly, this application proposes a method for visual recognition and inspection of automotive IBC products after they have been manufactured, based on the aforementioned system, comprising the following steps:

[0019] The visual module controls the feeding process to identify the placement posture of products to be inspected on the conveyor line.

[0020] Based on the posture data, the robot arm module is coordinated to grasp the product and transfer it to the inspection station fixed by the rotary table module;

[0021] The rotating stage module and the vision inspection module are coordinated to perform multi-angle scanning and defect detection and classification of products;

[0022] Based on the defect classification results, the robot arm module is controlled to perform the corresponding sorting and stacking operations, and the data traceability module is coordinated to bind the detection data with the product's unique identifier and upload it to the MES system.

[0023] For products classified as defective, the corresponding downstream linkage processing flow is triggered according to the defect level.

[0024] According to the technical solution provided in this application, the multi-angle scanning and defect detection classification of the product includes the following steps:

[0025] Based on the product model information, the corresponding 3D model and process data are called to divide the product surface into multiple functional areas, and each functional area is associated with its corresponding quality risk level.

[0026] Based on the quality risk level and the characteristics of the defects to be detected in each functional area, differentiated scanning control parameters are planned and executed for different functional areas. The scanning control parameters include rotation speed, scanning sampling frequency and illumination conditions.

[0027] Spatiotemporal synchronization and fusion processing are performed on 3D image data with different features collected from different functional areas, and macroscopic defect identification and high-precision microscopic morphology analysis are performed simultaneously.

[0028] Based on historical and real-time defect detection data, the detection stability of each functional area is evaluated, and the resource allocation strategy in subsequent scanning processes is adaptively adjusted accordingly.

[0029] According to the technical solution provided in this application, dividing the product surface into multiple functional areas includes the following steps:

[0030] Based on the assembly relationship and tolerance zone information of each component in the three-dimensional model, the contact surface constituting the assembly interface is automatically identified and extracted, and the contact surface and its surrounding preset buffer range are jointly defined as the high-precision detection area.

[0031] Based on the preset defect pattern library, the high-precision detection area is divided into sub-regions. Specifically, for the bolt fastening area, the thread engagement section, flange mating surface, and bolt head bearing surface are divided; for the PIN area, the needle body cylindrical surface and the needle tip conical surface are divided; and for the oil port area, the sealing groove and threaded hole are divided.

[0032] According to the technical solution provided in this application, after guiding the analysis of the micro-assembly area that is mechanically or technologically related to the macroscopic defect, the method further includes the following steps:

[0033] If the second channel detects an abnormal gap on the bolt flange mating surface or an overall tilt in the PIN area, it determines that there is a risk of local deformation caused by assembly stress.

[0034] Generate and execute deformation verification scanning action: control the robot arm module to grab the product from the turntable and apply simulated assembly load to the product in a preset force direction opposite to the suspected deformation direction;

[0035] While maintaining the simulated assembly load, the robot arm is controlled to place the product back onto the rotary table and re-clamp it. Then, a high-resolution local rescan with a fixed viewing angle is initiated on the high-precision detection area and its surrounding related areas where anomalies were previously identified.

[0036] The microscopic morphology data obtained by rescanning is compared with the data before loading. If the difference result exceeds the allowable elastic deformation threshold of the material in that area under the corresponding load, it is determined that the product has non-rigid deformation defects caused by residual stress or microcracks. The classification level of such defects is higher than that of static appearance defects.

[0037] According to the technical solution provided in this application, the step of guiding the analysis of the micro-assembly area that is mechanically or technologically related to the macroscopic defect includes the following steps:

[0038] Based on the product's three-dimensional digital model and assembly process documents, a correlation stress simulation analysis is initiated. The correlation stress simulation analysis is used to simulate the stress transmission path and concentration area caused by the macroscopic defects in the internal structure of the product under a given assembly or usage condition, and output at least one set of coordinates of high-probability anomalies located in the micro-assembly area.

[0039] The coordinate set of high-probability anomalies is spatially matched with the real-time analysis results of the graph neural network on the high-precision detection area to generate enhanced scanning and analysis instructions. The enhanced scanning and analysis instructions control the rotary table module and the vision detection module to perform local viewpoint supplementary scanning on the microscopic sub-regions corresponding to the coordinate set of high-probability anomalies, and instruct the graph neural network of the second channel to prioritize and deeply analyze the node features within the coordinate set range in order to quantitatively assess whether there is a microscopic assembly failure causally related to macroscopic defects.

[0040] According to the technical solution provided in this application, after the graph neural network of the second channel is instructed to preferentially and deeply analyze the node features within the coordinate set range, the method further includes the following steps:

[0041] The quantification results of the micro-assembly state output after the deep analysis of the graph neural network are compared with the theoretical deformation or stress distribution predicted by the correlation stress simulation analysis.

[0042] If the quantitative results and theoretical predictions show a consistent trend in the core region of the high-probability anomaly coordinate set, and the deviation amplitude does not exceed the preset tolerance, then it is determined that there is a verified mechanical correlation between the current macroscopic defects and microscopic anomalies.

[0043] The macroscopic defect type, its associated microscopic region location, and the quantitative deviation relationship between the two are included as verified defect association patterns and added to the preset defect pattern library.

[0044] In subsequent testing of similar products, when the first channel identifies the same macroscopic defect type again, it directly calls the verified defect association pattern. The second channel will then skip the general analysis process and directly perform a high-confidence targeted review on the specified microscopic area based on the associated location information stored in the pattern.

[0045] Compared with the prior art, the beneficial effects of this application are as follows:

[0046] I. The system achieves full automation and coverage of the inspection process: The system automatically identifies the product posture through the loading vision module, guides the robotic arm to accurately grasp and transfer the product to the rotary table; the central control system coordinates the synchronous movement of the rotary table and the vision inspection module, which can perform 360° three-dimensional scanning of the product without blind spots, completely avoiding the blind spots and efficiency bottlenecks of manual inspection, and ensuring the comprehensiveness and high speed of inspection.

[0047] Second, it improves the consistency and reliability of inspection: The entire process is automated and coordinated by the central control system, eliminating the subjectivity and fatigue of manual operation. The visual inspection module uses unified and objective 3D image data and AI algorithms to determine defects, ensuring that inspection standards remain consistent and significantly improving the reliability and quality stability of inspection results.

[0048] Third, it establishes a seamless link between quality data and production management: Through the data traceability module, the system can bind the unique identifier of each product with its complete inspection data and defect images in real time and upload them to the enterprise's MES system. This achieves structured storage and end-to-end traceability of quality data, providing a precise data foundation for production quality analysis, process improvement, and product lifecycle management, thus facilitating closed-loop quality control. Attached Figure Description

[0049] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0050] Figure 1 A schematic diagram of the automotive IBC product off-line appearance recognition and inspection system provided in this application;

[0051] Figure 2 A flowchart illustrating the steps of the automotive IBC product offline appearance recognition detection method provided in this application;

[0052] The text labels in the image represent:

[0053] 1. Material feeding vision module; 2. Central control system; 3. Rotary table module; 4. Vision inspection module; 5. Robotic arm module; 6. Data traceability module. Detailed Implementation

[0054] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0055] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0056] Example 1

[0057] As mentioned in the background section, this application proposes an automotive IBC product off-line appearance recognition and inspection system, such as... Figure 1 As shown, it includes:

[0058] The loading vision module 1 is used to identify the placement posture of the IBC products to be inspected on the conveyor line and output posture data.

[0059] Robotic arm module 5 is used to grasp the product based on the posture data and transfer it to the inspection station;

[0060] Rotary table module 3, located at the inspection station, is used to fix and rotate the product;

[0061] The visual inspection module 4 is used to scan the surface of the product during rotation, acquire and perform defect detection and classification based on three-dimensional image data;

[0062] The data traceability module 6 communicates with the central control system 2 and the enterprise MES system to bind the unique product identifier and test data.

[0063] The central control system 2 is communicatively connected to the loading vision module 1, the robotic arm module 5, the rotary table module 3, the vision inspection module 4, and the data traceability module 6, respectively. The central control system 2 is used to coordinate the operation of each module and, based on the results of defect detection and classification, control the robotic arm module 5 to perform sorting and stacking operations corresponding to the results.

[0064] The central control system 2 is configured for:

[0065] Based on the product model information, the corresponding 3D model and process data are called to divide the product surface into multiple functional areas, and each functional area is associated with its corresponding quality risk level.

[0066] The visual inspection module 4 is controlled to construct a dual-channel heterogeneous processing architecture during the scanning process. The first channel is loaded with a lightweight convolutional neural network to extract features and classify defects in the macroscopic appearance surface area. The second channel is loaded with a graph neural network based on point cloud data to construct a topology map of the sub-region point cloud of the micro-assembly area and quantitatively analyze assembly state anomalies and micro-morphological deviations.

[0067] A cross-channel attention guidance mechanism is established. When the first channel detects a specific type of defect in the macroscopic appearance area, it sends a signal to the second channel to guide it to focus on analyzing the microscopic assembly area that is mechanically or technologically related to the macroscopic defect. Based on the results of defect detection and classification, the robot arm module 5 is controlled to perform sorting and stacking operations corresponding to the results.

[0068] Specifically, the loading vision module 1 refers to the industrial camera (such as a 2D CCD or 3D contour camera) and its supporting image processing hardware and software installed upstream of the conveyor line. Its specific implementation for recognizing placement posture is as follows: when the conveyor line transports the IBC product (an integrated brake controller) to the predetermined area, the module photographs or scans the product, and calculates the product's position coordinates (X, Y) on the conveyor line and its rotation angle (θ) relative to the standard posture using image recognition algorithms (such as template matching or deep learning-based target detection algorithms). This information together constitutes posture data. The robotic arm module 5 is typically a six-axis industrial robot or a high-precision Cartesian coordinate robot, with an adaptive gripper (such as a pneumatic finger or vacuum suction cup) at its end. The implementation process is as follows: after receiving the posture data sent by the loading vision module 1, the central control system 2 generates robot motion trajectory instructions. The robotic arm moves to the calculated gripping point according to these instructions, controls the gripper to grasp the product with appropriate posture and force, and then smoothly and accurately moves the product to an independent inspection station along the planned path. The rotary table module 3 is fixedly installed on this inspection station. Its core is a precision rotary table driven by a servo motor. The rotary table is equipped with customized fixtures or clamps for reliably clamping and securing the product from the bottom or side after the robot arm places it on it. Its function of rotating the product is to drive the fixed product to perform precise stepping or continuous rotation under the command of the central control system 2 during the inspection process. The vision inspection module 4 is the core inspection unit of the system, and its implementation includes both hardware scanning and software analysis. The hardware part typically includes one or more 3D vision sensors (such as laser line scan sensors or structured light cameras); the software part is an image processing program running on an industrial computer or embedded processor. The implementation process is as follows: when the rotary table drives the product to rotate, the vision sensors continuously scan the product surface passing through their field of view, acquiring three-dimensional point cloud data or depth image sequences of the product surface. Subsequently, based on these three-dimensional image data, the software automatically identifies and classifies defects (such as scratches, dents, foreign objects, assembly errors, etc.) through built-in defect detection algorithms (such as comparing point clouds with CAD models, analyzing curvature anomalies, detecting edge defects, etc.). The data traceability module 6 can be a software middleware or a database interface program. Its implementation is as follows: This module communicates with the central control system 2 via a local area network or industrial bus to obtain the unique identifier of the current product (such as a scanned QR code or a system-generated serial number) and complete inspection result data (including defect images, types, locations, dimensions, etc.). Subsequently, it structurally binds this information and uploads the data packet to the enterprise's Manufacturing Execution System (MES) through a standard interface (such as Web Service, OPC UA), completing the association between quality data and production information. The central control system 2 is an industrial computer or PLC running configuration software or a customized control program.The system coordinates the operation of each module as follows: As the master station on the system bus, it sequentially sends trigger signals to the upward-feeding vision module 1 according to a preset work rhythm and logical sequence, receives status feedback from the robot arm module 5, controls the start / stop and rotation speed of the rotary table module 3, triggers the vision inspection module 4 to begin scanning, and receives its analysis results. The specific implementation of its sorting and stacking operation is as follows: Based on the defect classification results (such as qualified, minor defects, and severe defects) output by the vision inspection module 4, the central control system 2 plans different placement point coordinates for the robot arm module 5. For example, it controls the robot arm to move qualified products to the qualified product conveyor line, minor defective products to the re-inspection area, and severe defective products to the scrap bin, thereby achieving automatic sorting and stacking.

[0069] This implementation method integrates modules such as loading vision, robotic arms, rotary table, visual inspection, and central control to construct a collaborative automated inspection closed loop. The core technical problem it solves is the low efficiency, blind spots, and poor consistency inherent in traditional manual or fixed-camera inspection. Its technical principle lies in dynamic rotational scanning and centralized coordinated control. The rotary table moves the product, allowing the fixedly installed vision sensors to scan all outer surfaces, achieving inspection without blind spots. Through the unified scheduling of the central control system 2, precise synchronization and efficient connection of actions such as loading, positioning, scanning, analysis, and sorting are ensured, forming a smooth automated production line. The ultimate technical effect is to achieve efficient, comprehensive, and automated appearance quality inspection and sorting of automotive IBC products, significantly improving inspection cycle time and consistency, and enabling automatic recording of inspection data.

[0070] In a preferred embodiment, the central control system 2 is further configured to: trigger differentiated processing instructions linked to downstream production processes based on the results of defect detection and classification; wherein the differentiated processing instructions include: generating and sending targeted warning information to downstream assembly processes for products with minor defects, and generating and sending isolation and quality anomaly notifications for products with serious defects.

[0071] Specifically, a defect level judgment rule library is preset in the system or obtained from the MES system. After the vision inspection module 4 completes defect detection and classification, the central control system 2 not only controls the sorting and stacking according to the results, but also immediately triggers internal logic judgment to generate differentiated processing instructions. These instructions are standardized messages actively sent to the control system or MES server of the downstream process through the system's industrial communication interface (such as Ethernet, Profinet). The specific implementation steps for generating and sending targeted warning information to the downstream assembly process for products with minor defects can be as follows: Suppose the vision inspection module 4 determines that a product has a minor scratch that does not involve safety functions (classified as a minor defect). While controlling the robot arm to stack it in the processing area, the central control system 2 will generate a warning message containing the product's unique identifier (such as VIN code or serial number), the defect type, location image, and a prompt that confirmation is required on the final assembly line. This message is sent in real time to the terminal display or mobile device of the corresponding assembly station downstream through the workshop network. In this way, when the product flows to the station, the operator will receive a prompt in advance and perform targeted confirmation to avoid the defect flowing into subsequent processes or causing assembly problems. The specific implementation steps for generating and sending isolation and quality anomaly notifications for critically defective products are more stringent: Assuming a structural crack or missing critical assembly component is detected in a product (classified as a critical defect), the central control system 2, after controlling a robotic arm to place the product into a scrap bin for physical isolation, will immediately generate a more detailed quality anomaly report. This report, in addition to product identification and defect details, may also include relevant production batch and time information. This report will be sent simultaneously as a higher-priority message to the MES system and relevant personnel in the production quality management department (via email, SMS, or internal communication software), and may trigger an Andon system red light, thereby quickly notifying all parties to contain, investigate, and analyze the defect, initiating the quality traceability process.

[0072] This implementation significantly accelerates the processing speed of quality anomalies, enabling minor defects to be handled in a controlled manner and preventing problems from escalating; it also allows serious defects to be immediately isolated and root cause analysis to be initiated quickly, thereby improving the overall responsiveness and quality control level of the production system.

[0073] In a preferred embodiment, the central control system 2 is further connected to a barcode scanning module; the central control system 2 is used to instruct the barcode scanning module to read the product's unique identifier, and obtain the product model information from the MES system based on the unique identifier, and then call up the exclusive detection parameters and barcode placement rules that match the model.

[0074] Specifically, the barcode scanning module refers to an industrial barcode reader (such as a fixed QR code reader or RFID reader / writer) fixedly installed near the robotic arm's gripping position or at the entrance of the inspection station. The implementation is as follows: After the robotic arm grips the product, or before the product is placed on the rotary table, the central control system 2 sends a reading command to the barcode scanning module. The scanning module scans the unique product identifier pre-engraved or affixed to the product, which is typically a barcode, QR code, or RFID tag storing a serial number. The scanning module uploads the read identifier string (e.g., IBC-A2-20240527-0001) to the central control system 2. The implementation of obtaining product model information from the MES system based on the unique identifier is as follows: The central control system 2 has a pre-installed client program that communicates with the MES system. After obtaining the unique identifier, the program uses this identifier as a key index to initiate a request to the MES system through a database query interface (such as an SQL query or API call). The MES system retrieves the corresponding product model, production batch, and process version information based on the serial number in its production database, and returns the product model information (e.g., IBC-Pro-2024) to the central control system 2. The core personalized implementation step is to call up the exclusive inspection parameters and placement rules matching that model. The software of the central control system 2 maintains a model-parameter configuration database. Upon receiving a specific product model (e.g., IBC-Pro-2024), the system automatically loads a pre-set configuration for that model from this database. These configurations include, but are not limited to: exclusive inspection parameters: the standard 3D CAD model file used by the visual inspection module 4 for analysis (for comparison); defect judgment thresholds for specific structures of that model (e.g., allowable scratch depth, pin height tolerance); a list of coordinates for areas requiring focused scanning; and specific weight files required for the AI ​​recognition model. Exclusive placement rules: specific placement coordinates corresponding to different defect levels (e.g., for model A, slightly defective products are placed in frame 1; for model B, they are placed in frame 2); and the posture requirements during placement. After loading these dedicated configurations, the central control system 2 uses them to initialize the control parameters of the vision inspection module 4 and the robotic arm module 5, thereby ensuring that subsequent scanning, analysis and sorting actions are tailored to the specific product model.

[0075] This implementation method enables the same physical hardware system to automatically identify and switch testing standards and processing procedures for different models of IBC products without manual switching or prolonged downtime for adjustments. This greatly enhances the flexibility and mixed-flow production capacity of the production line, reduces downtime caused by product changes, and ensures testing accuracy and applicability for different products.

[0076] In a preferred embodiment, the visual inspection module 4 includes a 3D laser line scan camera and an AI image processing unit; the rotary table module 3 is a high-precision servo rotary table; the central control system 2 controls the rotary table module 3 and the 3D laser line scan camera to move synchronously, so as to realize 360° scanning of the product fixed on the rotary table module 3 without blind spots; the AI ​​image processing unit identifies the three-dimensional image data obtained by scanning, including the determination of macroscopic appearance defects of the product and the microscopic details and assembly status of bolts, pins, oil ports, and assembly parts.

[0077] Specifically, a 3D laser line scanning camera is a sensor that calculates the three-dimensional coordinates of an object's surface by emitting a laser beam onto the surface and capturing the deformation of the laser beam with a camera on one side. In this implementation, the camera is fixedly mounted on one side of the inspection station, with its laser beam plane approximately perpendicular to the rotation axis of the rotary table. As the product rotates, the laser beam sequentially scans different cross-sections of the product, and the camera continuously captures images, thus obtaining a series of cross-sectional contour point cloud data. By stitching these continuous cross-sectional point cloud data according to the rotation angle, the complete three-dimensional surface morphology of the product can be reconstructed. The AI ​​image processing unit can be an industrial control computer equipped with a GPU, where the software integrates deep learning frameworks (such as TensorFlow or PyTorch). In this implementation, the unit receives three-dimensional point cloud or depth image data from the 3D camera and calls a neural network model (such as a convolutional neural network CNN or a point cloud processing network PointNet++) pre-trained with a large number of defect samples to perform inference analysis on the data. The identification process, which includes judging macroscopic appearance defects of products as well as microscopic details and assembly status of bolts, pins, oil ports, and assembled components, is as follows: The AI ​​model is trained to perform multi-task identification simultaneously. For example, one branch network outputs the category and location of macroscopic defects such as scratches and dents; another branch network specifically analyzes the bolt area to determine whether the threads are complete, whether there is stripping, and whether the lower surface of the bolt head is fully fitted; for the pin area, it determines whether the pin body is bent or missing; for the oil port, it determines whether the threads are damaged and whether the sealing surface is flat. The rotary table module 3 is a high-precision servo rotary table. This means that the rotary table is driven by a servo motor and equipped with a high-resolution encoder for position feedback, enabling precise angular positioning (such as an accuracy of ±0.01°) and smooth speed control. The central control system 2 controls the rotary table module 3 to move synchronously with the 3D laser line scanning camera to achieve 360° scanning without blind spots, which is the key to collaborative work. Its implementation usually adopts hardware triggering or precise software timing. For example, the central control system 2 sends a command to the servo driver to make the rotary table start rotating at a constant angular velocity. Simultaneously, the central control system 2 sends an external trigger signal to the 3D laser line scanning camera. The frequency of this signal matches the rotation speed of the turntable, ensuring that the camera is triggered to acquire a frame of contour data for every tiny angle (such as 0.1°) the product rotates. Through this strict synchronization, it can be guaranteed that all the acquired contour data are continuous and uniform in angular coordinates, and finally seamlessly stitched into a complete and seamless 3D model, truly achieving 360° scanning without blind spots.

[0078] This implementation method improves the overall performance of the detection system: it achieves full coverage in terms of detection range, reaches the microscopic level in terms of detection accuracy, and realizes automatic classification of various complex defects in terms of detection intelligence, thereby meeting the stringent quality inspection requirements of high-end automotive safety components.

[0079] Example 2

[0080] Based on Example 1, this example proposes a method for visual recognition and detection of automotive IBC products after they have been manufactured, implemented using the system described above, as follows: Figure 2 As shown, it includes the following steps:

[0081] S1. Control the feeding vision module 1 to identify the placement posture of the products to be inspected on the conveyor line;

[0082] The specific implementation is as follows: The central control system 2 continuously monitors the status of the loading station through cyclic querying or interrupt triggering. When the sensor detects that the product has reached the designated position, the central control system 2 sends an image acquisition command to the loading vision module 1 (industrial camera). After the camera captures an image of the product, it runs its built-in image processing algorithm. This algorithm first determines the outline of the product in the image through edge detection or feature point matching, and then compares it with a pre-stored standard upright posture template to calculate the pixel coordinate offset of the product's center point and the rotation angle around the vertical axis. These calculation results (ΔX, ΔY, θ) are encapsulated as posture data and sent back to the central control system 2. This is the basis for all subsequent precise positioning operations.

[0083] S2. Based on the posture data, coordinate the robot arm module 5 to grab the product and transfer it to the inspection station fixed by the rotary table module 3;

[0084] Specifically, the coordination is manifested as follows: After receiving the attitude data, the central control system 2's path planning software calibrates and converts the world coordinate system of the conveyor line, the camera coordinate system, and the robot arm base coordinate system. It converts the image-based offsets and angles into precise three-dimensional coordinates (X, Y, Z) of the gripping point in the robot arm base coordinate system and the rotation angle that the end effector needs to compensate for. Then, the central control system 2 sends movement commands containing these target poses to the motion controller of the robot arm module 5. The robot arm executes these commands, moves to the calculated gripping point, and controls its end effector's adaptive gripper (such as a specific gripper selected according to the product model) to perform the gripping action. After confirming a secure grip, the robot arm smoothly transfers the product along a preset obstacle avoidance trajectory and places it onto the dedicated gripper of the rotary table module 3. The gripper of the rotary table module 3 then automatically locks under the command of the central control system 2, completing the product fixation.

[0085] S3, coordinating the rotary table module 3 and the vision inspection module 4, performs multi-angle scanning and defect detection and classification of the product;

[0086] Specifically, its implementation involves two levels: hardware coordination and software analysis. In terms of hardware coordination, the central control system 2 sends speed control commands to the servo driver of the rotary table module 3, instructing it to begin rotating at a uniform speed. Simultaneously, it sends a hardware trigger signal to the 3D laser line scan camera of the vision inspection module 4. The frequency of this signal is strictly synchronized with the rotational speed of the rotary table, ensuring that the cross-sectional contour is scanned every time the product rotates by a tiny angle (e.g., 0.1°). In terms of software analysis, the AI ​​image processing unit of the vision inspection module 4 receives and stitches these cross-sectional contours in real time, forming a complete 3D point cloud model. Subsequently, the AI ​​model (such as a trained deep learning network) loads the corresponding product standard CAD model for comparison, or directly performs feature analysis on the point cloud, identifying defects such as scratches, dents, missing parts, and abnormal screws. These defects are then automatically classified into different levels, such as "qualified," "minor defect" (Class A), and "serious defect" (Class B), according to preset rules (e.g., defect size, depth, and location). The detection and classification results are fed back to the central control system 2 in real time.

[0087] S4. Based on the defect classification results, control the robotic arm module 5 to perform the corresponding sorting and stacking operations, and coordinate with the data traceability module 6 to bind the detection data with the product's unique identifier and upload it to the MES system;

[0088] Specifically, this is a parallel processing process. On one hand, the central control system 2 immediately retrieves the corresponding target location coordinates from the stacking rule base based on the received defect classification results. For example, if the result is qualified, the target location is the qualified product exit conveyor belt; if it is a minor defect, it is the waiting buffer station location A; if it is a serious defect, it is the scrap recycling bin location B. The central control system 2 generates new motion instructions, directing the robotic arm to grab the product from the rotary table and move it to the designated target location for release. On the other hand, throughout the entire process, the data traceability module 6 runs in the background. It obtains the unique identifier of the product from the central control system 2 (which may be provided in advance by the barcode scanning module or automatically generated by the system according to the production cycle) and associates and binds this identifier with the complete inspection report uploaded by the vision inspection module 4 (including 3D point cloud data, defect image, type, coordinates, classification level, etc.) to form a structured data packet. Finally, the data traceability module 6 uploads the data packet to the designated database or interface of the enterprise manufacturing execution system (MES) through standard industrial communication protocols (such as TCP / IP, OPC UA) to complete data archiving.

[0089] S5. For products classified as defective, trigger the corresponding downstream linkage processing flow according to the defect level.

[0090] The implementation method is as follows: The central control system 2 is pre-configured with processing logic corresponding to different defect levels. For example, when a product is classified as having a minor defect, after sorting and stacking, the system will automatically generate a warning message containing the product serial number, a brief description of the defect, and a suggested handling method, and send it to the electronic Kanban board or operator terminal of the relevant assembly station downstream via the workshop network. When a product is classified as having a serious defect, in addition to physical isolation (placing it in the scrap bin), the system will generate a more detailed quality anomaly alarm. This alarm will not only be sent to the downstream station, but may also be sent simultaneously to the production quality manager's WeChat and email, and trigger a red alarm in the Andon system, requiring the quality engineer to intervene immediately.

[0091] This implementation method achieves end-to-end fully automated, traceable, and intelligently linked appearance quality inspection of automotive IBC products from start to finish. This not only significantly improves inspection efficiency and consistency and reduces labor costs, but more importantly, it establishes a closed loop for real-time quality feedback and control, significantly enhancing the overall quality control capability and response speed of the production line.

[0092] In a preferred embodiment, the multi-angle scanning and defect detection classification of the product includes the following steps:

[0093] Based on the product model information, the corresponding 3D model and process data are called to divide the product surface into multiple functional areas, and each functional area is associated with its corresponding quality risk level.

[0094] Specifically, after the central control system 2 obtains the current product model (e.g., IBC-Pro) through barcode scanning or MES, it automatically retrieves the 3D digital model (e.g., STEP format CAD file) and process data file for that model from its local or server database. The process data file defines the inspection requirements for each area in a structured form (e.g., XML, JSON). The system software first parses the 3D model and identifies different geometric feature surfaces. Then, combined with annotations in the process data (e.g., "bolt connection surface, critical safety item," "shell side, general appearance surface"), it logically segments the product surface in 3D space. For example, it automatically segments the top cover plane area, side wall curved surface area, bolt mounting boss area, PIN pin array area, main oil port end face area, etc. Next, the system assigns a quality risk level quantification value or label to each area according to the pre-set rules in the process data (e.g., high risk: level 3, medium risk: level 2, low risk: level 1). The risk level setting may be based on whether the area involves safety functions, assembly sealing, electrical conductivity, and the frequency and severity of defects in that area in historical quality data.

[0095] Based on the quality risk level and the characteristics of the defects to be detected in each functional area, differentiated scanning control parameters are planned and executed for different functional areas. The scanning control parameters include rotation speed, scanning sampling frequency and illumination conditions.

[0096] Specifically, the scanning planner in the central control system 2 dynamically generates a non-uniform scanning scheme for the current product based on the area-risk level mapping table obtained in the previous step and the physical characteristics of different defects (macroscopic scratches require comprehensive scanning, while microscopic threads require high-resolution detail scanning). This scheme includes a series of control instructions that vary over time (or rotation angle). For example, for low-risk macroscopic appearance areas, the first set of parameters is planned: control the rotary table to rotate at a relatively high speed (e.g., 30 RPM) at a constant speed, and control the 3D camera to scan at a standard frame rate (e.g., 1000 Hz) and standard laser power, aiming to quickly cover a large area. For high-risk bolt and pin areas, when the rotary table is about to bring the area into the camera's field of view, the second set of parameters is planned: control the rotary table to slow down to a low speed (e.g., 5 RPM) or even step rotation within a specific angle range, while controlling the 3D camera to switch to high-resolution mode (e.g., increase sampling to 2000 Hz, or enable multi-line scanning mode), and triggering auxiliary strip light sources to illuminate at a specific angle to eliminate shadows and obtain the clearest microscopic morphological details. Variations in lighting conditions are crucial for detecting minute scratches or dents on the oil port sealing surface.

[0097] Spatiotemporal synchronization and fusion processing are performed on 3D image data with different features collected from different functional areas, and macroscopic defect identification and high-precision microscopic morphology analysis are performed simultaneously.

[0098] Specifically, since the data sampling rate and resolution may differ in different areas, the system first interpolates and aligns all the scanned point cloud profile data to the same global coordinate system and the same angle sampling interval, based on precise timestamps and angle information fed back from the rotary table encoder, fusing them into a complete, but non-uniformly dense, 3D product point cloud model. Subsequently, the AI ​​image processing unit initiates parallel analysis tasks. One processing thread (or lightweight model) is responsible for quickly analyzing low-resolution, large-scale macroscopic point clouds, identifying larger scratches, dents, and other defects. Another dedicated high-precision processing thread (or specialized model) performs in-depth analysis only on high-risk area point cloud data obtained using high-resolution scanning, performing precise measurements such as thread profile fitting, PIN height and perpendicularity calculation, and sealing surface flatness evaluation.

[0099] Based on historical and real-time defect detection data, the detection stability of each functional area is evaluated, and the resource allocation strategy in subsequent scanning processes is adaptively adjusted accordingly.

[0100] Specifically, the system database continuously records the inspection results (presence or absence of defects) for each product and each functional area, as well as the confidence score from AI analysis. The optimization algorithms built into the central control system 2 (such as those based on Statistical Process Control, SPC) analyze this data periodically (e.g., after every 100 products inspected) or in real time. For example, if a sidewall curved surface area (low-risk) shows no defects in hundreds of consecutive inspections and maintains a consistently high confidence level, the system may, in subsequent scans, slightly reduce the scanning resolution or slightly increase the scanning speed for that area without affecting coverage, allocating the saved time to more critical areas. Conversely, if the defect rate of a specific bolt area for a particular model has recently increased, the system will automatically increase the risk level of that area in the scanning plan and may add additional lighting angles or scanning perspectives to capture more comprehensive information, ensuring no defects are missed.

[0101] The core technical problem addressed by this implementation method is how to balance detection coverage and detection accuracy within a limited detection cycle time, especially for different areas of varying importance on complex workpieces. The achieved technical effect is to optimize overall detection efficiency while ensuring or even improving the detection accuracy of key features, thus achieving the best balance between quality, efficiency, and cost. This method is particularly suitable for modern flexible production lines with high precision and high cycle time.

[0102] In a preferred embodiment, dividing the product surface into multiple functional areas includes the following steps:

[0103] Based on the assembly relationship and tolerance zone information of each component in the three-dimensional model, the contact surface constituting the assembly interface is automatically identified and extracted, and the contact surface and its surrounding preset buffer range are jointly defined as the high-precision detection area.

[0104] Specifically, after loading the product's 3D digital assembly model, the system software not only reads the geometric information but also analyzes the assembly constraints (such as fitting, alignment, and insertion) contained in the model. The algorithm traverses all components, automatically identifying surfaces with close mating relationships, such as the mating surfaces of valve blocks and end caps, the contact surfaces of bolt flanges and connected parts, and the contact surfaces of sealing rings and groove walls. These contact surfaces directly affect the product's sealing performance, connection strength, and functional realization. Simultaneously, the system reads the tolerance zone information of these mating relationships from the associated process data (such as flatness 0.05mm and positional tolerance Φ0.1mm). Based on this, the algorithm not only marks these contact surfaces themselves but also automatically extends a buffer zone around them (e.g., an area extending outwards by 2-5mm). The logic behind setting this buffer zone is that assembly stress or defects may affect the area adjacent to the contact surface, and a certain contextual area is required for feature analysis during detection. All areas defined in this way are collectively referred to as high-precision detection areas, which are the focus of subsequent microscopic analysis.

[0105] Based on the preset defect pattern library, the high-precision detection area is divided into sub-regions. Specifically, for the bolt fastening area, the thread engagement section, flange mating surface, and bolt head bearing surface are divided; for the PIN area, the needle body cylindrical surface and the needle tip conical surface are divided; and for the oil port area, the sealing groove and threaded hole are divided.

[0106] The simultaneous execution of macroscopic defect identification and high-precision microscopic morphology analysis includes the following steps:

[0107] A dual-channel heterogeneous processing architecture is constructed. The first channel is loaded with a lightweight convolutional neural network to extract features and classify defects after downsampling of the macroscopic appearance surface area. The second channel is loaded with a graph neural network based on point cloud data to construct a topology map of the sub-region point cloud of the high-precision detection area. The assembly state anomalies and micro-morphological deviations are quantitatively analyzed by iteratively updating the node features.

[0108] Establish a cross-channel attention guidance mechanism. When the first channel detects a specific type of defect or abnormal texture in the macroscopic appearance area, it sends a signal to the second channel to guide it to focus on analyzing the microscopic assembly area that is mechanically or technologically related to the macroscopic defect.

[0109] Specifically, the preset defect pattern library is a structured database that stores the correspondence between various common defects and their locations and forms. For example, for the bolt fastening area, the pattern library indicates that attention should be paid to whether the threads of the threaded engagement section are intact, whether the flange mating surface is flat and without gaps, and whether there are indentations or wear on the bolt head bearing surface. Therefore, the system will further segment the automatically identified bolt area into three sub-regions based on its three-dimensional geometric features. Similarly, for the PIN area, the system segments the pin body cylindrical surface (to check for bending and scratches) and the pin tip conical surface (to check for wear and deformation); for the oil port area, the system segments the sealing channel (to check the flatness of the channel bottom and whether the edges are chamfered) and the threaded hole (to check whether the threads are smooth and whether there are any broken teeth). This segmentation makes the subsequent AI analysis objectives extremely clear. Within the software framework of the AI ​​image processing unit, two relatively independent neural network models are actually deployed, running in parallel, but processing different data and tasks. The first channel loads a lightweight convolutional neural network (CNN). The implementation process is as follows: The system downsamples the low-resolution 3D data (or rendered 2D depth map) of the macroscopic appearance area (reducing the data volume to improve speed) and then inputs it into the CNN. This CNN model is specifically optimized for quickly identifying defects such as scratches, dents, and stains on large areas, and can complete feature extraction and classification within milliseconds. The second channel loads a graph neural network (GNN) based on point cloud data. Its implementation process is more complex: The system does not treat the fine point cloud data of the high-precision detection area and its sub-regions as an unordered set, but constructs a "topological graph" based on the spatial proximity of points. Each node in the graph represents a point or a local point cluster, and the edges represent connections. This GNN model can learn the complex local structure and global context in the point cloud by iteratively updating and transmitting information on the features of nodes and edges. It is particularly good at analyzing "assembly state anomalies". For example, by analyzing the point cloud map of the bolt area, it can determine whether the distribution of the thread segment point cloud conforms to the spiral pattern (to determine whether there is stripping), and whether the flange surface point cloud forms a flat plane (to determine gaps). When the first channel's CNN analyzes a macroscopic region (e.g., the side of the shell), it not only outputs defect classifications, but also, if it identifies a specific defect pattern (e.g., a radial fine crack texture), a specific activation pattern will be displayed in a feature map of a certain intermediate layer within it. The system is designed with an attention mapping module that can map this activation pattern back to a general region in three-dimensional space. Simultaneously, the system has a pre-set mechanical / process-related knowledge table, which defines that radial cracks on the shell side may be related to stress concentration caused by excessive bolt preload at the corresponding internal location. Therefore, the central control system 2 sends a guiding signal to the second channel's GNN, which contains the coordinates of the microscopic region that needs to be analyzed (i.e., the internal bolt mounting area corresponding to the crack location).After receiving the signal, the second channel's GNN will adjust its internal computational attention to prioritize and analyze the point cloud node features within the coordinate range in greater detail, thereby verifying whether macroscopic defects have caused associated microscopic assembly problems.

[0110] The technical problem solved by this embodiment is that traditional visual inspection methods for complex assemblies remain at the surface and isolated analysis level, making it difficult to deeply connect the intrinsic causal relationship between macroscopic anomalies and microscopic assembly conditions. It not only tells the user where the problem is, but also, to some extent, suggests why it might be faulty (e.g., macroscopic cracks indicate excessive local stress, thus guiding the inspection of related fasteners), making the inspection report more diagnostically valuable, providing direct clues for process improvement, and achieving a leap from defect detection to preliminary diagnosis.

[0111] In a preferred embodiment, after guiding the analysis to focus on the micro-assembly areas that are mechanically or technologically related to the macro-defect, the method further includes the following steps:

[0112] If the second channel detects an abnormal gap on the bolt flange mating surface or an overall tilt in the PIN area, it determines that there is a risk of local deformation caused by assembly stress.

[0113] Generate and execute deformation verification scanning action: control the robot arm module 5 to grab the product from the rotary table and apply simulated assembly load to the product in a preset force direction opposite to the suspected deformation direction;

[0114] While maintaining the simulated assembly load, the robot arm is controlled to place the product back onto the rotary table and re-clamp it. Then, a high-resolution local rescan with a fixed viewing angle is initiated on the high-precision detection area and its surrounding related areas where anomalies were previously identified.

[0115] The microscopic morphology data obtained by rescanning is compared with the data before loading. If the difference result exceeds the allowable elastic deformation threshold of the material in that area under the corresponding load, it is determined that the product has non-rigid deformation defects caused by residual stress or microcracks. The classification level of such defects is higher than that of static appearance defects.

[0116] Specifically, after the second channel's graph neural network (GNN) completes in-depth analysis of the high-precision detection area, it outputs not only defect labels but also some continuous quantitative indicators. For example, for the bolt flange mating surface, the GNN will fit and calculate an average gap value and gap non-uniformity; for the PIN pin area, it will calculate an overall tilt angle. The central control system 2 has preset warning thresholds for these indicators. If the average gap value is greater than the standard value but does not reach the direct rejection limit, or the gap non-uniformity is significant, or the overall tilt angle exceeds the tolerance, the system will not immediately classify it as a deterministic defect (because it may be measurement noise or a slight error). Instead, based on process knowledge (such as non-uniform gaps often caused by assembly off-center loading, which may contain residual stress), it will mark it as having a risk of local deformation due to assembly stress. This is a suspected risk marker, triggering the subsequent verification process. Based on the risk type and location, a preset simulated assembly load scheme is selected from the verification action library. For example, for a suspected single-sided gap on a flange face, this load might be applied by the end effector of a robotic arm, in a direction perpendicular to the flange face, to the side with the smaller gap with a brief, small force (e.g., 20N, lasting 1 second). The direction and magnitude of this force simulate the condition where the part might be corrected or further tightened during final assembly. In practice, the central control system 2 first instructs the rotary table fixture to release the product, and then directs the robotic arm to re-grasp the product. In the gripping state, the robotic arm's force control system (or force control simulated through a current loop) starts working, controlling the end effector to slowly contact a fixed force sensor in a predetermined direction or reach a preset motor current value, thereby precisely applying the simulated assembly load.

[0117] Specifically, while maintaining the simulated assembly load, the robot arm is controlled to place the product back onto the rotary table and re-clamp it. Then, a high-resolution local rescan with a fixed viewing angle is initiated on the previously identified high-precision detection area and its surrounding related areas, acquiring deformation data under load. This is a delicate operation: while maintaining the applied force (maintaining motor current), the robot arm needs to very smoothly place the product back onto the rotary table's fixture. The rotary table fixture needs to be designed to lock the product under pressure without changing its force posture. After the product is fixed, since the robot arm has moved away, the load applied to the product is maintained by the reaction force of the rotary table fixture. Subsequently, the system no longer performs a full rotational scan, but instead controls the 3D laser line scan camera to perform an ultra-high-resolution local scan only on the previously risky bolt flange mating surface or PIN pin array area, from one or several fixed optimal viewing angles, acquiring the local microscopic morphology of the product under load. The system software precisely registers and aligns the new point cloud data obtained under load with the original high-precision point cloud data of the same region in step S3 (before loading) in three-dimensional space. Then, it performs point-by-point or region-by-region differential calculations on the two point clouds. The calculated displacement vector field reflects the deformation of the region before and after loading. The system pre-stores the mechanical property parameters of the product material (such as aluminum alloy). Combined with the magnitude and direction of the applied load, a theoretical allowable elastic deformation threshold can be calculated (i.e., under this load, a normal product should only undergo small, recoverable elastic deformation, and its deformation should be within this threshold). If the actual deformation obtained from the differential calculation (especially non-uniform, abrupt deformation) significantly and continuously exceeds this allowable elastic deformation threshold, it indicates that the material or structure in the region has become abnormal. This usually means the presence of residual stress (internal stress puts the material in a critical state, and a slight external force will produce abnormal deformation) or microcracks (cracks open under external force, causing a sudden increase in local displacement). This type of defect is absolutely undetectable by static appearance inspection, and is therefore classified as a non-rigid deformation defect, which is of a higher grade than static appearance defects such as surface scratches and stains.

[0118] This implementation addresses a highly insidious yet extremely dangerous technical problem: how to detect residual stress, microcracks, or potential structural weaknesses within a product. These defects may appear perfectly normal when not under stress, but can lead to malfunctions when subjected to stress during subsequent assembly, transportation, or use. The system possesses the ability to detect latent defects, intercepting potential problems that might only surface at the customer's end before the product leaves the factory. This significantly improves the long-term reliability and safety margin of products, especially safety-critical components such as automotive brake controllers.

[0119] In a preferred embodiment, guiding the analysis to focus on micro-assembly areas that are mechanically or technologically related to the macro-defect includes the following steps:

[0120] Based on the product's three-dimensional digital model and assembly process documents, a correlation stress simulation analysis is initiated. The correlation stress simulation analysis is used to simulate the stress transmission path and concentration area caused by the macroscopic defects in the internal structure of the product under a given assembly or usage condition, and output at least one set of coordinates of high-probability anomalies located in the micro-assembly area.

[0121] The coordinate set of high-probability anomalies is spatially matched with the real-time analysis results of the graph neural network on the high-precision detection area to generate enhanced scanning and analysis instructions. The enhanced scanning and analysis instructions control the rotary table module 3 and the vision detection module 4 to perform local viewpoint supplementary scanning on the microscopic sub-regions corresponding to the coordinate set of high-probability anomalies, and instruct the graph neural network of the second channel to prioritize and deeply analyze the node features within the coordinate set range in order to quantitatively assess whether there is a microscopic assembly failure causally related to macroscopic defects.

[0122] Specifically, a lightweight online simulation engine is invoked. This engine has a built-in basic finite element analysis (FEA) solver or a pre-calculated reduced-order model. When the first channel's CNN identifies a macroscopic defect (e.g., a linear scratch approximately 10mm long with a specific direction found on the valve body sidewall), the central control system 2 performs two tasks: First, it accurately retrieves the 3D digital model of the product model (usually a meshed CAE model) from the local server; second, it retrieves the product's assembly process file from the MES or PLM system, which contains key process parameters such as bolt tightening torque sequences and seal press-fitting forces. Subsequently, the central control system 2 inputs the type, precise 3D location, and geometric features (such as scratch depth and direction) of the macroscopic defect as a boundary condition or initial damage into the online simulation engine. Simultaneously, the engine applies normal assembly loads (such as the preload of all bolts) to the model according to the process file. The simulation engine's task is to quickly calculate how the internal stress of the entire component will redistribute under assembly forces in the presence of this specific macroscopic defect. Its output is not a complete stress cloud map, but rather the coordinates of one or more microscopic regions with significantly higher stress concentrations than normal, obtained through algorithm optimization. For example, the simulation might show that a specific sidewall scratch causes stress to be transferred and concentrated at the root of a particular internal cavity mounting boss, thus marking that boss root region as a high-probability anomaly. Simultaneously, the second channel's GNN is performing routine parallel analysis on all the high-precision detection areas it is responsible for (such as the previously defined bolt area, PIN pin area, etc.). After the simulation is complete, the system obtains a "high-probability anomaly coordinate set" containing three-dimensional coordinates (e.g., the XYZ coordinates of several key points on the aforementioned mounting boss root surface). The system does not wait for the GNN to complete all analysis but immediately spatially matches this coordinate set with the coordinate system of the point cloud data being processed by the GNN. This is achieved through a pre-completion, precise simulation coordinate system-visual measurement coordinate system calibration. After matching, the system determines in real time which high-probability anomalies fall within a high-precision detection sub-region currently being processed by the GNN (for example, finding that the stress concentration point predicted by the simulation is located exactly within the flange mating surface sub-region of a bolt). Once a match is successful, the central control system 2 immediately generates a high-priority enhanced scan and analysis instruction. This instruction contains two core parts: 1) An enhanced scan instruction for the hardware: sent to the rotary table module 3 and the vision inspection module 4, requiring that within the current or next scan cycle, a local viewpoint supplementary scan be immediately performed on the local angle range where the successfully matched sub-region is located. This may mean controlling the rotary table to make an extra small back-and-forth movement and triggering the 3D camera to take close-up pictures of the local area with higher laser power or denser scan lines to obtain richer point cloud data with a higher signal-to-noise ratio than a regular scan.2) Deep analysis commands for the software: These are sent to the second-channel GNN, instructing it to prioritize allocating computing resources to graph nodes containing high-probability anomalies. The GNN adjusts its internal message passing mechanism, performing more rounds of iterative updates and more refined feature extraction on the features of these nodes, essentially conducting a diagnostic review of these "suspicious points." The hardware system responds to the commands, completing a rapid and targeted rescan. Newly acquired, higher-quality point cloud data is injected in real-time into the graph structure being processed by the GNN, updating the attributes of relevant nodes. Guided by attention, the GNN performs deep analysis on the target node and its neighborhood. Its quantitative evaluation is no longer a binary judgment of whether or not there is a defect, but a precise measurement of causal relationships. For example, for the aforementioned bolt flange mating surface, the GNN will focus on calculating whether the uniformity of the contact pressure distribution near the simulation prediction point and the gradient change of the micro-gap match the stress concentration trend predicted in the simulation. Through this in-depth analysis, the system can output a quantitative assessment result, such as macroscopic scratches leading to a 45% increase in the unevenness of contact pressure on the associated bolt surfaces, thereby scientifically confirming or ruling out the causal relationship between macroscopic and microscopic defects, providing strong evidence for quality root cause analysis.

[0123] This implementation method can explain the possible intrinsic connections between different defects from the perspective of mechanical principles, upgrading the inspection report from a list of phenomena to a preliminary failure analysis report, providing unprecedented data insights for precise improvement of production processes, and realizing the intelligent upgrade of quality control.

[0124] In a preferred embodiment, after instructing the graph neural network of the second channel to preferentially and deeply analyze the node features within the coordinate set range, the method further includes the following steps:

[0125] The quantification results of the micro-assembly state output after the deep analysis of the graph neural network are compared with the theoretical deformation or stress distribution predicted by the correlation stress simulation analysis.

[0126] If the quantitative results and theoretical predictions show a consistent trend in the core region of the high-probability anomaly coordinate set, and the deviation amplitude does not exceed the preset tolerance, then it is determined that there is a verified mechanical correlation between the current macroscopic defects and microscopic anomalies.

[0127] The macroscopic defect type, its associated microscopic region location, and the quantitative deviation relationship between the two are included as verified defect association patterns and added to the preset defect pattern library.

[0128] In subsequent testing of similar products, when the first channel identifies the same macroscopic defect type again, it directly calls the verified defect association pattern. The second channel will then skip the general analysis process and directly perform a high-confidence targeted review on the specified microscopic area based on the associated location information stored in the pattern.

[0129] Specifically, the system possesses two sets of data for the same microscopic region: one set is the quantification results of the microscopic assembly state after GNN deep analysis (this is a set of measured data, such as an array of actual gap values ​​and contact pressure distribution vectors at various points on the bolt flange surface); the other set is the theoretical deformation or stress distribution output by the simulation engine (this is a set of predicted data, such as the normal displacement cloud map or contact stress cloud map at various points on the flange surface calculated by simulation). The system initiates a comparison algorithm, which first performs rigorous registration and mapping of the two sets of data on a spatial grid to ensure that the physical quantities compared are at the same physical location. Then, the comparison algorithm calculates the trend correlation between the two in spatial distribution (for example, by calculating the cross-correlation coefficient between the two images to determine whether the measured high gap region coincides with the location of the simulation-predicted high stress region), and simultaneously calculates the absolute deviation amplitude of key feature parameters (such as maximum gap value and average pressure). The preset tolerance is an engineering-defined threshold that allows for the existence of simulation and measurement errors. For example, if simulation predicts that the stress concentration factor at a certain point should be 1.5, and actual measurement shows that the average gap in that area increases by 30%, with a consistent trend (both indicating a deterioration in the condition of that area), and the 30% increase is within the allowable measurement fluctuation range (e.g., tolerance of ±35%), then the verification is deemed successful. The verified mechanical correlation means that this inspection not only discovered a macroscopic scratch and a microscopic gap inconsistency, but also, through simulation-guided in-depth analysis, confirmed with high confidence a causal relationship between the two determined by assembly mechanics.

[0130] Specifically, the central control system 2 will add the macroscopic defect type, its associated microscopic region location, and the quantitative deviation relationship between the two as a verified defect association pattern to the preset defect pattern library. The system will generate a standardized structure to describe this new pattern, for example:

[0131] Pattern ID: Pattern_001;

[0132] Triggering conditions (macro): Linear scratch on the side wall of the housing, length > 5 mm, direction at 45 ± 10 degrees to the main axis;

[0133] Related micro-regions: third bolt mounting base, flange mating surface;

[0134] Related characteristics: The measured contact pressure non-uniformity increased by 25-50% compared to the baseline;

[0135] Confidence level: High (verified through simulation and measurement);

[0136] This structure is added as a new record to the preset defect pattern library. This library transforms from a static, historically experience-based database into a dynamically growing, causal-logical intelligent knowledge base.

[0137] In subsequent production, when the CNN of the first channel identifies a macroscopic scratch that highly matches the trigger condition of Pattern_001 again, the system will no longer initiate the time-consuming full-range simulation analysis. Instead, it will directly search the pattern library and call Pattern_001. The central control system 2 will perform steps such as "initiating correlation stress simulation analysis" and directly issue instructions to the GNN of the second channel: "Immediately focus on checking the flange mating surface of the third bolt mounting seat, and use a dedicated analysis model to evaluate whether its contact pressure non-uniformity is within the expected range of 25-50%." This is equivalent to providing the GNN with a highly specific checklist. The second channel can therefore skip the general, traversal analysis of all micro-areas of the entire product and go straight to the point, performing a rapid and high-confidence targeted review of known risk points. The focus of the review shifts from discovering unknown anomalies to verifying whether the expected correlation holds, and the analysis speed and the certainty of quality judgment are greatly improved.

[0138] This implementation significantly improves subsequent inspection efficiency by eliminating a large amount of simulation calculations and general search analysis; it also greatly improves the accuracy and consistency of defect correlation judgment because subsequent judgments are based on verified physical laws and measured data patterns, rather than fluctuations that may be introduced with each recalculation. This allows the inspection system to continuously evolve with the accumulation of production data, making its inspection strategy increasingly precise and efficient.

[0139] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A vehicle IBC product off-line appearance recognition and inspection system, characterized in that, include: The loading vision module (1) is used to identify the placement posture of the IBC products to be inspected on the conveyor line and output posture data; The robotic arm module (5) is used to grasp the product according to the posture data and transfer it to the inspection station; A rotary table module (3) is located at the testing station and is used to fix and rotate the product; The visual inspection module (4) is used to scan the surface of the product during rotation, acquire and perform defect detection and classification based on three-dimensional image data; The data traceability module (6) communicates with the central control system (2) and the enterprise MES system to bind the product's unique identifier and testing data; The central control system (2) is connected to the loading vision module (1), the robot arm module (5), the rotary table module (3), the vision inspection module (4), and the data traceability module (6) respectively. The central control system (2) is used to coordinate the operation of each module and control the robot arm module (5) to perform sorting and stacking operations corresponding to the results based on the defect detection and classification results. The central control system (2) is configured for: Based on the product model information, the corresponding 3D model and process data are called to divide the product surface into multiple functional areas, and each functional area is associated with its corresponding quality risk level. The visual detection module (4) is controlled to construct a dual-channel heterogeneous processing architecture during the scanning process. The first channel is loaded with a lightweight convolutional neural network to extract features and classify defects in the macroscopic appearance surface area. The second channel is loaded with a graph neural network based on point cloud data to construct a topology map of the sub-region point cloud of the micro-assembly area and quantitatively analyze assembly state anomalies and micro-morphological deviations. Establish a cross-channel attention guidance mechanism. When the first channel detects a specific type of defect in the macroscopic appearance area, it sends a signal to the second channel to guide it to focus on analyzing the microscopic assembly area that is mechanically or technologically related to the macroscopic defect. Based on the results of defect detection and classification, the robot arm module (5) is controlled to perform sorting and stacking operations corresponding to the results.

2. The automotive IBC product off-line appearance recognition and inspection system according to claim 1, characterized in that, The central control system (2) is also configured to: trigger differentiated processing instructions linked with downstream production links based on the results of defect detection and classification; wherein, the differentiated processing instructions include: generating and sending targeted warning information to downstream assembly links for products with minor defects, and generating and sending isolation and quality anomaly notifications for products with serious defects.

3. The automotive IBC product off-line appearance recognition and inspection system according to claim 1, characterized in that, The central control system (2) is also connected to a barcode scanning module; the central control system (2) is used to instruct the barcode scanning module to read the product's unique identifier, and obtain the product model information from the MES system based on the unique identifier, and then call the exclusive detection parameters and barcode placement rules that match the model.

4. The automotive IBC product off-line appearance recognition and inspection system according to claim 1, characterized in that, The visual inspection module (4) includes a 3D laser line scanning camera and an AI image processing unit; the rotary table module (3) is a high-precision servo rotary table; the central control system (2) controls the rotary table module (3) and the 3D laser line scanning camera to move synchronously, so as to realize 360° scanning of the product fixed on the rotary table module (3) without dead angles; the AI ​​image processing unit identifies the three-dimensional image data obtained by scanning, including the determination of macroscopic appearance defects of the product and the microscopic details and assembly status of bolts, pins, oil ports, and assembly parts.

5. A method for visual inspection of automotive IBC products after production line completion, implemented based on the system described in any one of claims 1 to 4, characterized in that, Includes the following steps: The control loading vision module (1) identifies the placement posture of the products to be inspected on the conveyor line; Based on the posture data, the robot arm module (5) is coordinated to grab the product and transfer it to the inspection station fixed by the rotary table module (3); The rotating stage module (3) and the vision inspection module (4) are coordinated to perform multi-angle scanning and defect detection and classification of the product; Based on the defect classification results, the control robot module (5) performs the corresponding sorting and stacking operation, and coordinates the data traceability module (6) to bind the detection data with the product's unique identifier and upload it to the MES system; For products classified as defective, the corresponding downstream linkage processing flow is triggered according to the defect level.

6. The automotive IBC product off-line appearance recognition and detection method according to claim 5, characterized in that, The process of multi-angle scanning and defect detection and classification of products includes the following steps: Based on the product model information, the corresponding 3D model and process data are called to divide the product surface into multiple functional areas, and each functional area is associated with its corresponding quality risk level. Based on the quality risk level and the characteristics of the defects to be detected in each functional area, differentiated scanning control parameters are planned and executed for different functional areas. The scanning control parameters include rotation speed, scanning sampling frequency and illumination conditions. Spatiotemporal synchronization and fusion processing are performed on 3D image data with different features collected from different functional areas, and macroscopic defect identification and high-precision microscopic morphology analysis are performed simultaneously. Based on historical and real-time defect detection data, the detection stability of each functional area is evaluated, and the resource allocation strategy in subsequent scanning processes is adaptively adjusted accordingly.

7. The automotive IBC product off-line appearance recognition and detection method according to claim 6, characterized in that, The process of dividing the product surface into multiple functional areas includes the following steps: Based on the assembly relationship and tolerance zone information of each component in the three-dimensional model, the contact surface constituting the assembly interface is automatically identified and extracted, and the contact surface and its surrounding preset buffer range are jointly defined as the high-precision detection area. Based on the preset defect pattern library, the high-precision detection area is divided into sub-regions. Specifically, for the bolt fastening area, the thread engagement section, flange mating surface, and bolt head bearing surface are divided; for the PIN area, the needle body cylindrical surface and the needle tip conical surface are divided; and for the oil port area, the sealing groove and threaded hole are divided.

8. The automotive IBC product off-line appearance recognition and detection method according to claim 7, characterized in that, After guiding the analysis to focus on the micro-assembly areas that are mechanically or technologically related to the macro-defect, the method further includes the following steps: If the second channel detects an abnormal gap on the bolt flange mating surface or an overall tilt in the PIN area, it determines that there is a risk of local deformation caused by assembly stress. Generate and execute deformation verification scanning action: control the robot arm module (5) to grab the product from the rotary table and apply simulated assembly load to the product in a preset force direction opposite to the suspected deformation direction; While maintaining the simulated assembly load, the robot arm is controlled to place the product back onto the rotary table and re-clamp it. Then, a high-resolution local rescan with a fixed viewing angle is initiated on the high-precision detection area and its surrounding related areas where anomalies were previously identified. The microscopic morphology data obtained by rescanning is compared with the data before loading. If the difference result exceeds the allowable elastic deformation threshold of the material in that area under the corresponding load, it is determined that the product has non-rigid deformation defects caused by residual stress or microcracks. The classification level of such defects is higher than that of static appearance defects.

9. The automotive IBC product off-line appearance recognition and detection method according to claim 7, characterized in that, The method of guiding the analysis of the micro-assembly areas that are mechanically or technologically related to the macro-defect includes the following steps: Based on the product's three-dimensional digital model and assembly process documents, a correlation stress simulation analysis is initiated. The correlation stress simulation analysis is used to simulate the stress transmission path and concentration area caused by the macroscopic defects in the internal structure of the product under a given assembly or usage condition, and output at least one set of coordinates of high-probability anomalies located in the micro-assembly area. The coordinate set of high-probability anomalies is spatially matched with the real-time analysis results of the graph neural network on the high-precision detection area to generate enhanced scanning and analysis instructions. The enhanced scanning and analysis instructions control the rotary table module (3) and the vision detection module (4) to perform local viewpoint supplementary scanning on the micro sub-regions corresponding to the coordinate set of high-probability anomalies, and instruct the graph neural network of the second channel to prioritize and deeply analyze the node features within the coordinate set range in order to quantitatively assess whether there is a micro assembly failure causally related to macroscopic defects.

10. The automotive IBC product off-line appearance recognition and detection method according to claim 9, characterized in that, After instructing the graph neural network of the second channel to preferentially and deeply analyze the node features within the coordinate set range, the method further includes the following steps: The quantification results of the micro-assembly state output after the deep analysis of the graph neural network are compared with the theoretical deformation or stress distribution predicted by the correlation stress simulation analysis. If the quantitative results and theoretical predictions show a consistent trend in the core region of the high-probability anomaly coordinate set, and the deviation amplitude does not exceed the preset tolerance, then it is determined that there is a verified mechanical correlation between the current macroscopic defects and microscopic anomalies. The macroscopic defect type, its associated microscopic region location, and the quantitative deviation relationship between the two are included as verified defect association patterns and added to the preset defect pattern library. In subsequent testing of similar products, when the first channel identifies the same macroscopic defect type again, it directly calls the verified defect association pattern. The second channel will then skip the general analysis process and directly perform a high-confidence targeted review on the specified microscopic area based on the associated location information stored in the pattern.

Images