Multi-vehicle-type collinear-oriented automobile flexible production line AI visual global fool-proof system and method

Abstract

The invention provides an AI visual global fool-proof system and method for a multi-vehicle-type collinear automobile flexible production line. Belongs to the technical field of computer vision. The method comprises the steps of performing vehicle type, process and fool-proof knowledge graph construction on a multi-vehicle-type collinear vehicle flexible production line, and generating knowledge graph data; and developing a vehicle type real-time identification module according to the knowledge graph data, and constructing a vehicle type real-time identification system. By constructing the vehicle type, the process and the fool-proof knowledge graph, real-time and accurate recognition of the vehicle type is realized, vehicle type switching signals can be quickly and accurately captured during multi-vehicle-type mixed-line production, a corresponding fool-proof strategy is dynamically loaded at a millisecond level, the problems of production interruption and missing detection caused by tedious vehicle type switching operation and slow system response are avoided, and the production efficiency is improved. And the production efficiency and continuity of the flexible production line are greatly improved.

Description

Technical Field

[0001] This invention proposes an AI-based global vision-based error-proofing system and method for flexible automotive production lines with multiple vehicle models on the same line, belonging to the field of computer vision technology. Background Technology

[0002] In the field of new energy vehicle manufacturing, flexible production lines with multiple models operating on the same line have become the mainstream production mode in order to improve equipment utilization and reduce production costs. However, existing AI visual error-proofing systems have revealed many problems that urgently need to be solved when dealing with this complex production environment.

[0003] Currently, error-proofing logic is often hard-coded and tied to work orders for specific vehicle models. When switching models, manual loading of configuration files or system restarts are required, which is not only cumbersome and error-prone but also interrupts the production process, severely impacting production efficiency. Moreover, existing vision systems are often limited to single-station inspection and lack the ability to verify global consistency across stations and processes. This makes it difficult to detect chain errors such as "incorrect assembly in the preceding sequence and uninspected subsequent sequence," leading to frequent quality problems.

[0004] Meanwhile, the installation position, angle, and obstruction relationship of the same part vary greatly on different vehicle models, making it difficult for universal inspection models to adapt to these changes, resulting in a significant drop in accuracy. In addition, there is a delay of several seconds between the MES issuing the new model instruction and the mistake-proofing system taking effect. In high-paced production lines, these few seconds can cause several vehicles to be missed in inspection, posing a great risk to production quality.

[0005] Against this backdrop, while the market offers systems claiming to possess features such as "flexible production lines," "AI visual error prevention," and "multi-model support," most are simply polling multiple independent error prevention modules, failing to integrate vehicle model recognition, BOM parsing, error prevention strategy generation, and global consistency verification into a unified AI vision engine with millisecond-level response. Therefore, developing a novel AI visual global error prevention method is urgently needed. Summary of the Invention

[0006] This invention provides an AI-based visual global error-proofing system and method for flexible automotive production lines with multiple vehicle models on the same line, to solve the problems mentioned in the background section above:

[0007] This invention proposes an AI-based global visual error-proofing method for flexible automotive production lines with multiple vehicle models on the same line. The method includes:

[0008] S1. Construct a knowledge graph of vehicle models, processes, and error-proofing for a flexible automotive production line with multiple vehicle models on the same line, and generate knowledge graph data; develop a real-time vehicle model recognition module based on the knowledge graph data, and build a real-time vehicle model recognition system.

[0009] S2. Capture vehicle model switching signals. When a new vehicle model switching signal is captured, load the error prevention strategy corresponding to the new vehicle model and generate error prevention strategy data for the new vehicle model. Based on the error prevention strategy data for the new vehicle model, adaptively adjust the parameters of the vision inspection equipment at each workstation on the flexible production line to obtain the adjusted vision inspection equipment parameter data.

[0010] S3. Using the adjusted visual inspection equipment parameter data, multi-source visual signals are acquired at each workstation of the flexible production line to obtain the original visual inspection signal data and acquisition timestamp data; noise filtering and image enhancement processing are then performed to generate high-quality visual inspection signal data.

[0011] S4. Perform global consistency verification across workstations and processes based on high-quality visual inspection signal data to obtain local inspection result data for each workstation and global consistency verification result data; perform error chain analysis on the local inspection result data for each workstation, identify chain errors, and generate chain error identification data.

[0012] S5. By collecting timestamp data, perform three-dimensional spatial modeling of the flexible production line to generate three-dimensional spatial model data of the production line; combine chain error identification data with the three-dimensional spatial model data of the production line to perform working condition correlation analysis to generate error-proof working condition correlation data; perform top plate drop risk prediction on error-proof working condition correlation data to obtain automotive assembly error risk prediction data.

[0013] S6. Based on the prediction data of automobile assembly error risk, a weighted risk index is processed to generate a production line assembly risk index; based on the production line assembly risk index, risk warning levels are classified to generate AI vision global error prevention warning data for automobile flexible production lines.

[0014] The AI-based global vision-based error-proofing system for flexible automotive production lines with multiple vehicle models on the same line, proposed in this invention, includes:

[0015] One or more processors;

[0016] Memory, used to store one or more programs;

[0017] Wherein, when the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method described in any one of the above.

[0018] The beneficial effects of this invention are as follows: By constructing a knowledge graph of vehicle models, processes, and error prevention, real-time and accurate vehicle model identification is achieved. This enables rapid and accurate capture of vehicle model switching signals during multi-model mixed-line production, dynamically loading corresponding error prevention strategies at the millisecond level. This avoids production interruptions and missed inspections caused by cumbersome vehicle model switching operations and slow system response, greatly improving the production efficiency and continuity of flexible production lines. The global consistency verification function across workstations and processes acts like a "panoramic X-ray vision" system for the production line, comprehensively and meticulously identifying chain errors such as "incorrect assembly in the preceding sequence and unchecked in the subsequent sequence." This effectively compensates for the shortcomings of existing vision systems' localized detection, significantly reducing product quality risks and improving product pass rates, thus providing a foundation for the development of new energy vehicles. High-quality production is provided with a solid guarantee; the parameters of the visual inspection equipment are adaptively adjusted based on the knowledge graph, enabling the inspection model to better adapt to the installation differences of the same part on different car models, greatly improving the feature generalization ability, ensuring inspection accuracy, reducing misjudgments and omissions caused by insufficient model accuracy, and making mistake-proof inspection more accurate and reliable; combining the collected timestamp data to perform 3D spatial modeling of the production line and working condition correlation analysis can more accurately predict the risk of automobile assembly errors and generate detailed mistake-proof warning data, providing production personnel with timely and effective decision-making basis, helping to take measures to prevent risks in advance, further enhancing the safety and stability of the production line, and giving new vitality to intelligent manufacturing. Attached Figure Description

[0019] Figure 1 This is a diagram illustrating the steps of the method described in this invention. Detailed Implementation

[0020] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0021] One embodiment of the present invention, such as Figure 1 As shown, an AI-based global visual error-proofing method for flexible automotive production lines with multiple vehicle models on the same line is described, the method comprising:

[0022] S1. Construct a knowledge graph of vehicle models, processes, and error prevention for a flexible automotive production line with multiple vehicle models on the same line, generating knowledge graph data containing vehicle features, process flow, and error prevention rules; develop a real-time vehicle model recognition module based on the knowledge graph data, thereby constructing a real-time vehicle model recognition system;

[0023] S2. Based on the real-time vehicle model recognition system, capture vehicle model switching signals. When a new vehicle model switching signal is captured, dynamically load the error prevention strategy corresponding to the new vehicle model in milliseconds according to the knowledge graph data to generate new vehicle model error prevention strategy data. At the same time, based on the new vehicle model error prevention strategy data, adaptively adjust the parameters of the visual inspection equipment at each workstation on the flexible production line to obtain the adjusted visual inspection equipment parameter data.

[0024] S3. Using the adjusted visual inspection equipment parameter data, multi-source visual signals are acquired at each workstation of the flexible production line to obtain raw visual inspection signal data and acquisition timestamp data; noise filtering and image enhancement processing are performed on the raw visual inspection signal data to generate high-quality visual inspection signal data.

[0025] S4. Perform global consistency verification across workstations and processes based on high-quality visual inspection signal data to obtain local inspection result data and global consistency verification result data for each workstation. Perform error chain analysis on the local inspection result data of each workstation using the global consistency verification result data to identify chain errors where the preceding sequence is incorrectly assembled and the subsequent sequence is not inspected, and generate chain error identification data.

[0026] S5. By collecting timestamp data, perform three-dimensional spatial modeling of the flexible production line to generate three-dimensional spatial model data of the production line; combine chain error identification data with the three-dimensional spatial model data of the production line to perform working condition correlation analysis to generate error-proof working condition correlation data; perform top plate drop risk prediction (analogous to the risk caused by automobile assembly error, actually the risk of automobile assembly error) on the error-proof working condition correlation data to obtain automobile assembly error risk prediction data.

[0027] S6. Based on the prediction data of automobile assembly error risk, a weighted risk index is processed to generate a production line assembly risk index; based on the production line assembly risk index, risk warning levels are classified to generate AI vision global error prevention warning data for automobile flexible production lines.

[0028] The working principle and effects of the above technical solution are as follows: By constructing a knowledge graph of vehicle models, processes, and error prevention, and developing a real-time recognition system, the adaptability flexibility of multi-model co-production is significantly improved. New model error prevention strategies are loaded in milliseconds, and equipment parameters are adaptively adjusted, significantly improving error prevention response efficiency during model switching. After noise filtering and image enhancement processing, visual inspection accuracy is significantly improved, providing reliable data support for subsequent verification. Cross-station global consistency verification combined with chain error analysis effectively enhances global error prevention capabilities, reducing localized missed detections due to incorrect preceding assembly and unchecked subsequent assembly. Combining operational condition correlation analysis and risk prediction with the production line's three-dimensional spatial model, assembly error risks can be identified in advance, avoiding losses such as part damage and production line downtime caused by incorrect assembly. Early warning levels are divided based on risk indices, improving the targeting of error prevention warnings and reducing production interruptions caused by excessive intervention.

[0029] In one embodiment of the present invention, S1 includes:

[0030] S11. Collect basic data of multiple vehicle models (including vehicle appearance features, body structure parameters, and component models), flexible production line process flow data (including process sequence, workstation responsibilities, and assembly standards), and historical error prevention case data (including error types, detection methods, and corrective measures) to generate a multi-source production line error prevention basic dataset.

[0031] S12. Perform data cleaning (removing duplicate and missing data), data labeling (adding classification labels to vehicle features, process nodes, and error prevention rules) and data association (establishing mapping relationships between vehicle models and adapted processes, and between processes and corresponding error prevention rules) on the multi-source production line error prevention basic dataset to generate standardized error prevention knowledge source data.

[0032] S13. Based on standardized error-proofing knowledge source data, use graph database technology (e.g., Neo4j) to define core entities (vehicle model, process, workstation, error-proofing rule) and relationships between entities (e.g., vehicle model A - adaptation - process B, process B - to be executed - error-proofing rule C), and construct a knowledge graph data containing vehicle features, process flow, and error-proofing rules.

[0033] S14. Extract key visual features (such as body outline, window shape, and chassis interface position) of each vehicle model from the knowledge graph data, and generate a vehicle model feature template library using a feature clustering algorithm (such as K-Means). The vehicle model feature template library is used to provide feature benchmarks for vehicle model recognition.

[0034] S15. Based on the vehicle feature template library, generate a real-time vehicle recognition module using a deep learning recognition algorithm; adapt the real-time vehicle recognition module to the visual acquisition hardware (such as industrial cameras, lenses, and light sources) at the production line entrance, debug the module's recognition accuracy and response speed, and build a real-time vehicle recognition system.

[0035] The working principle and effects of the above technical solution are as follows: By collecting multi-source data such as vehicle models, processes, and historical error-proofing cases, and after cleaning and removing invalid information, labeling and mapping, the reliability of the basic data is significantly improved, avoiding errors in error-proofing logic caused by data clutter or missing data. A knowledge graph is constructed using a graph database to clearly outline the relationship between vehicle models, processes, and error-proofing rules, enhancing the systematic nature of error-proofing knowledge and providing a solid foundation for rapid strategy matching. Key visual features of vehicle models are extracted to build a template library. A recognition module is developed using deep learning and adapted to hardware debugging, significantly improving the accuracy of vehicle model recognition and ensuring timely response, avoiding misjudgment of vehicle models or recognition delays that could affect production line changeover efficiency.

[0036] In one embodiment of the present invention, S15 includes:

[0037] Based on the vehicle model feature template library, we select a deep learning recognition algorithm adapted to industrial scenarios (e.g., improve YOLOv8 and optimize the detection capabilities of small targets and occluded scenes). We use multi-angle and multi-light vehicle image samples derived from the template library (including front / side / top views of each vehicle model, covering normal, shadow, and dust conditions) to train the model, adjust the parameters (e.g., network learning rate, number of iterations, etc.), and generate an initial vehicle model recognition model.

[0038] The initial vehicle recognition model is functionally integrated by adding an image preprocessing submodule (to achieve size normalization, color correction, and noise filtering), a feature matching submodule (to compare input image features with template library features), and a result output submodule (to output vehicle ID and recognition confidence), forming a complete real-time vehicle recognition module.

[0039] Analyze the communication protocols (e.g., GigEVision, USB3Vision) of the visual acquisition hardware (industrial cameras, lenses, light sources) at the production line entrance, develop the communication interface between the real-time vehicle recognition module and the hardware, execute the module's control of the hardware's acquisition parameters (e.g., trigger frequency, exposure time) and image stream reception, and complete the communication adaptation between the two.

[0040] The adapted vehicle model real-time recognition module was tested using a test image set with multiple vehicle models and multiple interference scenarios (including 10+ vehicle models and 5 typical working conditions). The recognition accuracy (set threshold ≥ 99.5%) and response latency (set threshold ≤ 100ms) were statistically analyzed. If the target was not met, the model parameters or hardware parameters were adjusted until the target requirements were met.

[0041] By linking the debugged vehicle model real-time recognition module with the visual acquisition hardware, configuring the module to automatically start, retry in case of anomalies, and record recognition logs, a complete process of image acquisition, preprocessing, recognition, and result output is formed, thus constructing a vehicle model real-time recognition system.

[0042] The working principle and effects of the above technical solution are as follows: Using an improved YOLOv8 and training with multi-angle, multi-lighting samples, coupled with an image preprocessing submodule to optimize image quality, significantly improves the recognition capability under interference conditions such as workshop shadows and dust, avoiding misjudgments and missed judgments in small targets or occluded scenes. Integrating submodules such as feature matching and result output makes the recognition process more complete, further improving the reliability of the recognition results. Developing a dedicated interface for the hardware communication protocol enables precise control of the module's acquisition parameters and efficient reception of image streams, reducing recognition interruptions caused by incompatible software and hardware, and enhancing system stability. Through multi-vehicle, multi-scenario testing and debugging, the recognition accuracy is stabilized above 99.5%, and the response latency is controlled within 100ms, ensuring recognition efficiency and accuracy during vehicle model switching and avoiding disruptions to the production line rhythm due to recognition lag. Configuring automatic start and abnormal retry functions forms a complete process, reducing manual intervention costs.

[0043] In one embodiment of the present invention, S2 includes:

[0044] S21. Deploy the vehicle model real-time recognition system to the entrance station of the flexible production line, configure the system and the production line MES (Manufacturing Execution System) real-time data interaction interface, receive the production line workpiece flow signals in real time, and generate real-time data stream of vehicle model flow;

[0045] S22. Based on the real-time data stream of vehicle model flow, set a vehicle model switching trigger threshold (for example, if the recognition result of 3 consecutive frames of images is inconsistent with the current vehicle model, it is determined to be a switch), capture the new vehicle model switching signal through the signal comparison algorithm, and generate vehicle model switching trigger signal data.

[0046] S23. Based on the new model ID in the model switching trigger signal data, call the query interface of the knowledge graph data, match the error prevention rules (such as the detection accuracy requirements of key components and the defect judgment criteria) corresponding to the model in milliseconds, adapt the process nodes and related workstation information, and generate new model error prevention strategy data.

[0047] S24. Analyze the detection requirements (e.g., detection frame rate ≥ 25fps, recognition accuracy ≥ 0.98) in the new vehicle model error prevention strategy data, and convert them into parameter adjustment requirements for visual inspection equipment (e.g., camera exposure time, lens focal length, light source brightness, algorithm detection threshold), and generate a list of visual equipment parameter requirements.

[0048] S25. Based on the list of vision equipment parameter requirements, send parameter adjustment instructions to the vision inspection equipment (camera, light source controller, image processor) at each workstation of the flexible production line via an industrial bus (e.g., EtherCAT), automatically calibrate the working status of the equipment, and obtain the adjusted vision inspection equipment parameter data.

[0049] S26. Verify the validity of the adjusted visual inspection equipment parameter data (collect 100 test images to verify whether the detection accuracy and frame rate meet the standards). If they do not meet the standards, iterate the parameter adjustment process again until the error prevention strategy requirements are met.

[0050] The working principle and effects of the above technical solution are as follows: Real-time linkage between the recognition system and the MES system ensures accurate reception of workpiece flow signals. Combined with a switching judgment rule based on continuous 3-frame comparison, the accuracy of vehicle model switching recognition is significantly improved, avoiding mismatch issues caused by single-frame misjudgments. Based on the new vehicle model ID, a knowledge graph is invoked to quickly match the error-proofing rules, achieving millisecond-level strategy loading and completely resolving the vacuum problem of delayed error-proofing measures during vehicle model switching. Error-proofing detection requirements are transformed into a list of equipment parameters, and the vision equipment at each workstation is automatically calibrated via the EtherCAT bus. This not only reduces the tediousness of manual adjustments but also improves the accuracy of parameter adaptation, avoiding parameter deviations caused by manual operation. Closed-loop verification with 100 frames of image validation ensures that equipment parameters fully match the error-proofing requirements of the new vehicle model, preventing detection failure due to substandard parameters.

[0051] In one embodiment of the present invention, S3 includes:

[0052] S31. Based on the adjusted visual inspection equipment parameter data, configure the acquisition mode of each workstation's visual equipment (e.g., continuous acquisition for key workstations and trigger acquisition for auxiliary workstations), set the acquisition frequency (≥30fps for key workstations and ≥15fps for auxiliary workstations) and timestamp synchronization rules (error with production line PLC clock ≤10ms), and generate visual signal acquisition configuration data.

[0053] S32. Based on the visual signal acquisition configuration data, start the visual inspection equipment at each workstation, acquire image signals (such as the installation status of parts and the tightening of screws) and video signals (such as the dynamic flow of processes) during the assembly process of the production line, and record the acquisition timestamp corresponding to each frame of signal to generate the original visual inspection signal and timestamp associated dataset.

[0054] S33. For the original visual detection signal and the image signal in the timestamp associated dataset, a multi-scale Gaussian filtering algorithm is used to remove environmental noise (such as dust and light source reflection), and a median filtering algorithm is used to eliminate salt and pepper noise, generating denoised visual detection signal data.

[0055] S34. For low-contrast images in the denoised visual detection signal data, an adaptive histogram equalization algorithm is used to improve the clarity of local details. For overexposed / underexposed images, a Gamma correction algorithm is used to adjust the brightness to generate enhanced visual detection signal data.

[0056] S35. The denoising and enhancement processing results are fused, and qualified signals are selected through image quality evaluation indicators (such as peak signal-to-noise ratio PSNR≥35dB, structural similarity SSIM≥0.9) to generate high-quality visual detection signal data, and the corresponding acquisition timestamp association information is retained.

[0057] The working principle and effects of the above technical solution are as follows: The acquisition mode is configured differently according to the importance of each workstation. High-frequency continuous acquisition is performed at key workstations, while trigger-based acquisition is used at auxiliary workstations. This ensures that critical details such as screw tightening are not missed, while avoiding resource waste caused by ineffective acquisition. The synchronization error between the timestamp and the production line PLC clock is controlled within 10ms, significantly improving data time consistency and laying a solid foundation for subsequent cross-workstation verification. Multi-scale Gaussian filtering and median filtering are used for noise reduction, accurately eliminating interference from dust, reflections, and salt-and-pepper noise on the image, avoiding missed detection of assembly defects caused by noise. Adaptive histogram equalization and Gamma correction optimize the image, making low-contrast and abnormally exposed image details clearer, greatly improving the accuracy of subsequent inspections. Qualified data is screened using indicators of PSNR≥35dB and SSIM≥0.9 to ensure high-quality input visual signals and prevent inferior data from dragging down the overall error-proofing analysis effect.

[0058] In one embodiment of the present invention, step S4 includes:

[0059] S41. Combining the process flow sequence in the knowledge graph data (e.g., chassis assembly -- body welding -- parts installation -- final inspection), define global consistency verification rules. The global consistency verification rules include cross-station data consistency rules (e.g., for parts that have passed the inspection of the preceding station, the subsequent station must match the same part identifier) ​​and cross-process logical consistency rules (e.g., the parts installation process cannot be entered if the welding process is not completed), and generate a global verification rule library.

[0060] S42. Based on high-quality visual inspection signal data, perform local inspection on the assembly results of each station. The local inspection includes using a target detection algorithm (e.g., Faster R-CNN) to identify the presence of parts, using a size measurement algorithm (e.g., subpixel edge detection) to verify assembly accuracy, and using a defect classification algorithm (e.g., ResNet) to determine whether there is deformation / misalignment, and generating local inspection result data for each station (including qualified / unqualified markings, defect type, and inspection time).

[0061] S43. Call the global verification rule library, concatenate the local detection result data of each workstation according to the process time order, compare the cross-workstation data correlation (e.g., whether the part X model detected by workstation A is consistent with the part X model detected by workstation B) and the cross-process logic rationality (e.g., whether the detection time of process C is later than the completion time of process B), and generate global consistency verification result data (including consistency / inconsistency identifier, inconsistent workstation pair, and violation rule number).

[0062] S44. For inconsistent data in the global consistency verification results, trace the process flow path corresponding to the collection timestamp, and analyze whether there is a chain problem of undetected assembly errors in the preceding station (e.g., station A missed detection of part X being installed backwards) and misjudgment in the subsequent station due to errors in the preceding station (e.g., station B judged itself to have an assembly error because part X was installed backwards).

[0063] S45. Using error propagation path analysis algorithms (such as path tracing based on directed graphs), determine the starting station of the chain error, the type of error root cause (such as missed detection or misjudgment), and the subsequent stations involved in the propagation, and generate chain error identification data (including error chain ID, starting station, propagation path, and error type).

[0064] The working principle and effects of the above technical solution are as follows: Global verification rules are established based on the process flow, and the inspection results of each workstation are compared sequentially according to the process. This verifies both the consistency of component identification across workstations and the logical rationality of process flow, effectively avoiding the loophole of local inspection focusing only on itself and ignoring the connections between preceding and subsequent parts. Local inspection employs multiple algorithms to accurately identify the presence or absence of components, assembly precision, and defects, significantly improving the accuracy of single-workstation inspection. For inconsistent data detected, the flow path is traced through timestamps, and the error propagation chain is analyzed using directed graph algorithms. This accurately locates the initial error workstation and the root cause type, preventing subsequent workstations from being misjudged due to previous errors, reducing unnecessary rework and confusion of responsibility. For example, if a previously missed component is installed backwards, subsequent workstations will not misjudge it as their own assembly problem, and the rework entry point can be quickly found.

[0065] In one embodiment of the present invention, step S5 includes:

[0066] S51. Collect physical dimension data of the flexible production line (e.g., workstation length / width, equipment installation coordinates, conveyor track spacing), equipment motion parameters (e.g., conveyor belt speed, robotic arm working radius, lifting platform travel) and workpiece size data (e.g., body length, width and height of various car models, component assembly coordinates), and combine them with the real-time status of the production line corresponding to the timestamp of the collection (e.g., workpiece position, equipment start and stop status) to generate basic data for 3D modeling of the production line.

[0067] S52. Use 3D modeling software (such as Unity3D) to construct a scene based on the basic data of the 3D modeling of the production line: restore the workstation layout, equipment shape, and workpiece model, add motion logic (such as the dynamic flow of the conveyor belt and the robotic arm simulating assembly actions), and generate 3D space model data of the production line that is mapped 1:1 to the actual production line (supporting coordinate positioning and status query).

[0068] S53. Match the error station ID and error occurrence coordinates in the chain error identification data with the station coordinates and equipment model in the production line 3D space model data, associate the production line operating parameters when the error occurs (such as assembly speed, visual inspection frequency, workpiece flow interval), and generate error-proof working condition association data (including the 3D coordinates of the error location, associated equipment parameters, and error occurrence timestamp).

[0069] S54. Construct an assembly error risk prediction model, specifically: using error types (e.g., missing critical components, misaligned non-critical components), frequency of occurrence, and associated equipment status (e.g., equipment failure rate) in the error-proofing condition associated data as input features, and setting risk weights (safety risk weight 0.4, quality risk weight 0.3, efficiency risk weight 0.3) in combination with the adverse consequences caused by historical errors (e.g., rework time, scrap cost, safety hazard level).

[0070] S55. Use a gradient boosting tree algorithm (e.g., XGBoost) to train a risk prediction model, output the risk probability (e.g., the risk probability of missing engine parts is 0.95) and the degree of impact (e.g., causing production line to stop for 2 hours) for each error, and generate automobile assembly error risk prediction data.

[0071] The working principle and effects of the above technical solution are as follows: Data such as the physical dimensions of the production line, equipment parameters, and real-time status are collected. A 1:1 three-dimensional model is constructed using Unity3D to accurately restore the workstation layout, equipment movement, and workpiece shape. Even conveyor belt flow and robotic arm movements can be dynamically simulated, significantly improving the intuitiveness of error location and avoiding the difficulty of finding specific problem points based solely on textual data. The workstations and coordinates of chain-like errors are matched with the three-dimensional model, and operational parameters such as assembly speed and inspection frequency are associated, ensuring that errors no longer exist in isolation and providing a clear view of the complete working condition when an error occurs, reducing biases in cause analysis caused by fragmented information. The XGBoost algorithm, combined with safety, quality, and efficiency weights, is used to train a risk model that can accurately calculate the risk probability and impact of each error. For example, it can predict the high risk of missing critical components and the potential downtime, avoiding batch rework or safety hazards caused by sudden failures.

[0072] In one embodiment of the present invention, S51 includes:

[0073] S511. Analyze the data categories required for 3D modeling of flexible production lines, determine the specific data collection items for physical dimension data (workstation dimensions, equipment coordinates, etc.), equipment motion parameters (conveyor belt speed, robotic arm radius, etc.), and workpiece dimension data (vehicle body, component coordinates, etc.), and generate a data collection list.

[0074] S512. Based on the generated data collection list, various types of raw data are collected using corresponding tools, including: physical dimension data is measured using a laser rangefinder and total station, equipment motion parameters are exported from the production line PLC / MES system, and workpiece size data is obtained through a 3D scanner, generating multiple types of raw datasets.

[0075] S513, interface with the production line PLC clock system, synchronously collect timestamps, record the production line status at the time of each raw data collection (such as the specific coordinates of the workpiece on the track, the start and stop status of the robotic arm), and generate a real-time production line status dataset with timestamps.

[0076] S514. Associate and match the various types of original datasets with the real-time production line status dataset with timestamps, unify the data format (e.g., coordinates are unified to the Cartesian coordinate system, and parameter units are unified to the international standard units), remove invalid data (e.g., measurement values ​​that are outside the reasonable range), and generate the basic data for 3D modeling of the production line.

[0077] The working principle and effects of the above technical solution are as follows: First, clearly define the data categories required for modeling and formulate a collection list to accurately cover core items such as physical dimensions, equipment movement, and workpiece dimensions, avoiding the problem of missing key data during collection and improving the targeting of data collection. For different data types, select appropriate tools: laser rangefinders and total stations ensure accurate physical dimension measurements, while 3D scanners acquire workpiece data that is more realistic, improving the accuracy of raw data from the source and reducing errors caused by manual measurement or single tools. Synchronize the production line PLC clock to record timestamps and real-time status, ensuring that each data point corresponds to the workpiece position, equipment start / stop, and other operating conditions at the time of collection, avoiding the isolated problem of data being disconnected from the actual production line status. Finally, associate the data, unify the format, and remove invalid values, such as unifying the coordinates to the Cartesian coordinate system and filtering out out-of-range measurements, ensuring consistent basic data specifications and avoiding model deviations caused by data disorder or distortion during subsequent modeling.

[0078] In one embodiment of the present invention, S513 includes:

[0079] Organize the communication protocol of the production line PLC clock system (e.g., ModbusTCP, Profinet), configure the data interaction interface parameters (e.g., IP address, port number, transmission rate), establish a stable communication link with the PLC clock system, obtain standard clock signals, and generate clock synchronization reference data;

[0080] Based on clock synchronization reference data, timestamp synchronization modules are configured for various types of raw data acquisition nodes, and timestamp generation rules are set (accurate to milliseconds) to ensure that the time deviation between the acquisition action and the timestamp record is ≤5ms, and a raw data association table with timestamps is generated.

[0081] Determine the real-time status parameters of the production line that need to be recorded (workpiece track coordinates, robot arm start / stop status, conveyor belt running speed, etc.), and collect status information in real time through workstation sensors (such as photoelectric sensors, position encoders) and MES system interfaces to generate raw data of the real-time status of the production line.

[0082] The raw real-time status data of the production line is matched with the raw data association table with timestamps according to the collection time, and the status information corresponding to the status data is supplemented. The status information includes the identification of the collection equipment and the workstation number. Invalid data with abnormal timestamps are removed, and a real-time status dataset of the production line with timestamps is generated.

[0083] The working principle and effects of the above technical solution are as follows: By streamlining the PLC clock communication protocol and configuring interface parameters, a stable communication link is established to obtain a standard clock signal. Then, a synchronization module is equipped to each acquisition node, controlling the deviation between the acquisition action and the timestamp within 5ms, significantly improving the accuracy of the timestamp and avoiding errors in working condition matching caused by time misalignment during subsequent data association. After clarifying key status parameters such as workpiece coordinates and equipment start / stop, real-time acquisition by workstation sensors and the MES system ensures comprehensive capture of production line status information, reducing incomplete working condition restoration due to missing status data. The status data is precisely matched with the original data with timestamps at specific times, supplementing information such as equipment identification and workstation numbers, while eliminating invalid data with abnormal timestamps. This ensures that the generated dataset is both complete and reliable, preventing invalid data from interfering with the accuracy of subsequent modeling.

[0084] In one embodiment of the present invention, step S6 includes:

[0085] S61. Define the risk index weighting rule, which is as follows: Based on the impact level (safety risk: high / medium / low, quality risk: high / medium / low, efficiency risk: high / medium / low) in the automobile assembly error risk prediction data, assign weighting coefficients (high risk coefficient 1.0, medium risk coefficient 0.6, low risk coefficient 0.2), and introduce error occurrence frequency coefficients (single occurrence coefficient 1.0, repeated occurrence ≥3 times coefficient 1.5), and generate a risk weighting coefficient table;

[0086] S62. Based on the risk weighting coefficient table, calculate the risk probability in each automobile assembly error risk prediction data by weighting (weighted risk value = risk probability × impact level coefficient × frequency coefficient), summarize the weighted risk values ​​of all errors, obtain the overall assembly risk index of the production line, and generate production line assembly risk index data (including total risk index and sub-indices of each risk type).

[0087] S63. Referring to industry error-proofing standards and historical production line operation data, set risk warning level thresholds: low risk (total index < 30), medium risk (30 ≤ total index < 60), high risk (60 ≤ total index < 90), and emergency risk (total index ≥ 90). Compare the current production line assembly risk index with the thresholds to determine the corresponding warning level.

[0088] S64. Generate AI vision global error prevention warning data for the flexible automotive production line based on the warning level. The global error prevention warning data includes a warning level identifier, risk details that trigger the warning (error type, risk index contribution ratio), and corresponding handling suggestions (low risk: continuous monitoring, medium risk: manual re-inspection, high risk: workstation suspension, emergency risk: production line shutdown).

[0089] S65. Synchronize the error prevention warning data to the production line central control system (e.g., SCADA system), workstation operation terminal, and management personnel's mobile terminal to visualize the warning information. The visualization of the warning information includes realizing the warning information through pop-up windows on the central control screen, workstation light prompts (red / yellow / green corresponding to different levels), and mobile APP push notifications. At the same time, trigger linkage control (e.g., automatically suspend the corresponding workstation when there is a high risk, and trigger an emergency stop of the production line when there is an emergency risk) to complete the global error prevention closed loop.

[0090] The working principle and effects of the above technical solution are as follows: Weighting rules are set based on the impact levels of safety, quality, and efficiency, as well as the frequency of errors. Corresponding coefficients are assigned to different risks, making the risk index calculation more closely reflect the actual impact and significantly improving the accuracy of risk assessment, avoiding misjudgments caused by relying on a single indicator. The weighted risk values ​​are aggregated to obtain the total index and sub-indices, clearly presenting the specific composition of production line risks. Four-level early warning thresholds are defined based on industry standards and historical data, with differentiated handling suggestions for different indices—continuous monitoring for low-risk and workstation suspension for high-risk. This avoids excessive intervention for small risks that slows down production, while preventing serious consequences from uncontrolled large risks, thus improving the targeted nature of early warning intervention. Early warning data is synchronized to the central control unit, workstation terminals, and managers' mobile devices, providing multi-channel visual reminders through large-screen pop-ups, light prompts, and APP push notifications. Combined with the linked control of automatic workstation suspension for high-risk and emergency risk shutdown, a complete mistake-proof closed loop is formed. This allows relevant personnel to respond quickly and promptly prevents the spread of risks, avoiding batch rework, equipment damage, or even safety accidents due to untimely handling.

[0091] One embodiment of the present invention provides an AI-based visual global error-proofing system for flexible automotive production lines with multiple vehicle models on the same line, comprising:

[0092] One or more processors;

[0093] Memory, used to store one or more programs;

[0094] Wherein, when the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method described in any one of the above.

[0095] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A global AI-based visual error-proofing method for flexible automotive production lines with multiple vehicle models on the same line, characterized in that: The method includes: S1. Construct a knowledge graph of vehicle models, processes, and error-proofing for a flexible automotive production line with multiple vehicle models on the same line, and generate knowledge graph data; develop a real-time vehicle model recognition module based on the knowledge graph data, and build a real-time vehicle model recognition system. S2. Capture vehicle model switching signals. When a new vehicle model switching signal is captured, load the error prevention strategy corresponding to the new vehicle model and generate error prevention strategy data for the new vehicle model. Based on the error prevention strategy data for the new vehicle model, adaptively adjust the parameters of the vision inspection equipment at each workstation on the flexible production line to obtain the adjusted vision inspection equipment parameter data. S3. Using the adjusted visual inspection equipment parameter data, multi-source visual signals are acquired at each workstation of the flexible production line to obtain the original visual inspection signal data and acquisition timestamp data; noise filtering and image enhancement processing are then performed to generate high-quality visual inspection signal data. S4. Perform global consistency verification across workstations and processes based on high-quality visual inspection signal data to obtain local inspection result data for each workstation and global consistency verification result data; perform error chain analysis on the local inspection result data for each workstation, identify chain errors, and generate chain error identification data. S5. By collecting timestamp data, perform three-dimensional spatial modeling of the flexible production line to generate three-dimensional spatial model data of the production line; combine chain error identification data with the three-dimensional spatial model data of the production line to perform working condition correlation analysis to generate error-proof working condition correlation data; perform top plate drop risk prediction on error-proof working condition correlation data to obtain automotive assembly error risk prediction data. S6. Based on the prediction data of automobile assembly error risk, a weighted risk index is processed to generate a production line assembly risk index; based on the production line assembly risk index, risk warning levels are classified to generate AI vision global error prevention warning data for automobile flexible production lines.

2. The AI-based global vision-based error prevention method for flexible automotive production lines with multiple vehicle models on the same line, as described in claim 1, is characterized in that... S1 includes: S11. Collect basic data of multiple vehicle models, flexible production line process flow data and historical error prevention case data to generate a multi-source production line error prevention basic dataset. S12. Perform data cleaning, data labeling and data association on the multi-source production line error prevention basic dataset to generate standardized error prevention knowledge source data; S13. Based on standardized error-proofing knowledge source data, use graph database technology to define core entities and relationships between entities, and construct knowledge graph data; S14. Extract key visual features of each vehicle model from knowledge graph data, and generate a vehicle model feature template library using a feature clustering algorithm. The vehicle model feature template library is used to provide feature benchmarks for vehicle model recognition. S15. Based on the vehicle feature template library, generate a real-time vehicle recognition module using a deep learning recognition algorithm; adapt the real-time vehicle recognition module to the visual acquisition hardware at the production line entrance to build a real-time vehicle recognition system.

3. The AI-based global vision-based error prevention method for flexible automotive production lines with multiple vehicle models on the same line, as described in claim 1, is characterized in that... S2 includes: S21. Deploy the vehicle model real-time recognition system to the entrance station of the flexible production line, configure the system and the production line MES real-time data interaction interface, receive the production line workpiece flow signal in real time, and generate real-time data stream of vehicle model flow. S22. Based on the real-time data stream of vehicle model flow, set the vehicle model switching trigger threshold, capture the new vehicle model switching signal through the signal comparison algorithm, and generate vehicle model switching trigger signal data; S23. Based on the new vehicle ID in the vehicle switching trigger signal data, call the query interface of the knowledge graph data to match the error prevention rules, adapted process nodes and associated workstation information of the vehicle in milliseconds, and generate new vehicle error prevention strategy data. S24. Analyze the detection requirements in the new vehicle's error-proofing strategy data, transform them into parameter adjustment requirements for the vision inspection equipment, and generate a list of vision equipment parameter requirements. S25. Based on the list of vision equipment parameter requirements, send parameter adjustment instructions to the vision inspection equipment at each workstation of the flexible production line via the industrial bus, automatically calibrate the working status of the equipment, and obtain the adjusted vision inspection equipment parameter data. S26. Verify the validity of the adjusted visual inspection equipment parameter data. If it does not meet the standard, iterate the parameter adjustment process again until the error prevention strategy requirements are met.

4. The AI-based global vision-based error prevention method for flexible automotive production lines with multiple vehicle models on the same line, as described in claim 1, is characterized in that... The S3 includes: S31. Based on the adjusted visual inspection equipment parameter data, configure the acquisition mode of each workstation's visual equipment, set the acquisition frequency and timestamp synchronization rules, and generate visual signal acquisition configuration data. S32. Based on the visual signal acquisition configuration data, start the visual inspection equipment at each workstation, acquire image signals and video signals during the assembly process of the production line, and record the acquisition timestamp corresponding to each frame of signal to generate the original visual inspection signal and timestamp associated dataset. S33. For the original visual detection signal and the image signal in the timestamp associated dataset, a multi-scale Gaussian filtering algorithm is used to remove environmental noise, and a median filtering algorithm is used to eliminate salt-and-pepper noise, generating denoised visual detection signal data. S34. For low-contrast images in the denoised visual detection signal data, an adaptive histogram equalization algorithm is used to improve the clarity of local details. For overexposed / underexposed images, a Gamma correction algorithm is used to adjust the brightness to generate enhanced visual detection signal data. S35. The denoising and enhancement processing results are fused, qualified signals are selected through image quality assessment indicators, high-quality visual detection signal data is generated, and the corresponding acquisition timestamp association information is retained.

5. The AI-based global vision-based error prevention method for flexible automotive production lines with multiple vehicle models on the same line, as described in claim 1, is characterized in that... The S4 includes: S41. Combine the process flow sequence in the knowledge graph data to define global consistency verification rules and generate a global verification rule library; S42. Based on high-quality visual inspection signal data, perform local inspection on the assembly results of each station and generate local inspection result data for each station. S43. Call the global verification rule library, connect the local detection result data of each workstation in the order of process time, compare the correlation of cross-workstation data and the rationality of cross-process logic, and generate global consistency verification result data. S44. For inconsistent data in the global consistency verification results, trace the process flow path corresponding to the collection timestamp, and analyze whether there is a chain problem of undetected assembly errors in the preceding workstation and misjudgment in the subsequent workstation due to errors in the preceding workstation. S45. Using the error propagation path analysis algorithm, determine the starting station of the chain error, the type of the error root cause, and the subsequent stations involved in the propagation, and generate chain error identification data.

6. The AI-based global vision-based error prevention method for flexible automotive production lines with multiple vehicle models on the same line, as described in claim 1, is characterized in that... The S5 includes: S51. Collect physical dimension data, equipment motion parameters and workpiece size data of the flexible production line, and combine them with the real-time status of the production line corresponding to the collection timestamp to generate basic data for 3D modeling of the production line. S52. Use 3D modeling software to construct a scene from the basic data of the 3D modeling of the production line, and generate 3D spatial model data of the production line that is mapped 1:1 to the actual production line. S53. Match the error station ID and error occurrence coordinates in the chain error identification data with the station coordinates and equipment model in the production line 3D space model data, associate the production line operating parameters when the error occurs, and generate error-proof working condition association data. S54. Construct an assembly error risk prediction model, specifically: using error type, frequency of occurrence, and associated equipment status in the error-proofing condition associated data as input features, and setting risk weights in combination with the adverse consequences caused by historical errors; S55. Use the gradient boosting tree algorithm to train the risk prediction model, output the risk probability and impact of each error, and generate automobile assembly error risk prediction data.

7. The AI-based global vision-based error prevention method for flexible automotive production lines with multiple vehicle models on the same line, as described in claim 6, is characterized in that... S51 includes: S511. Analyze the data categories required for 3D modeling of flexible production lines, determine the specific data collection items for physical dimension data, equipment motion parameters, and workpiece size data, and generate a data collection list. S512. Based on the generated data collection list, use the corresponding tools to collect various types of raw data and generate multiple types of raw datasets; S513, connects to the production line PLC clock system, synchronously collects timestamps, records the production line status at the time of each raw data collection in real time, and generates a real-time production line status dataset with timestamps. S514. Associate and match the various types of original datasets with the real-time production line status dataset with timestamps, unify the data format, remove invalid data, and generate basic data for 3D modeling of the production line.

8. The AI-based global error-proofing method for flexible automotive production lines with multiple vehicle models on the same line, as described in claim 7, is characterized in that... S513 includes: Organize the communication protocol of the production line PLC clock system, configure the data interaction interface parameters, establish a stable communication link with the PLC clock system, obtain standard clock signals, and generate clock synchronization reference data. Based on clock synchronization reference data, timestamp synchronization modules are configured for various types of raw data acquisition nodes, timestamp generation rules are set, and raw data association tables with timestamps are generated. Determine the real-time status parameters of the production line that need to be recorded, collect status information in real time through workstation sensors and MES system interfaces, and generate raw data of the real-time status of the production line. The raw data of the real-time status of the production line is matched with the raw data association table with timestamps according to the collection time, the status information corresponding to the status data is supplemented, invalid data with abnormal timestamps is removed, and a real-time status dataset of the production line with timestamps is generated.

9. The AI-based global vision-based error prevention method for flexible automotive production lines with multiple vehicle models on the same line, as described in claim 1, is characterized in that... The S6 includes: S61. Define the risk index weighting rules and generate a risk weighting coefficient table; S62. Based on the risk weighting coefficient table, the risk probability in each automobile assembly error risk prediction data is weighted and calculated. The weighted risk values ​​of all errors are summarized to obtain the overall assembly risk index of the production line and generate the production line assembly risk index data. S63. Refer to industry error-proofing standards and historical production line operation data to set risk warning level thresholds, compare the current production line assembly risk index with the thresholds to determine the corresponding warning level; S64. Generate AI visual global error-proofing early warning data for flexible automotive production lines based on the early warning level; S65. Synchronize the error prevention warning data to the production line central control system, workstation operation terminal and management personnel mobile terminal to visualize the warning information, and trigger linkage control to complete the global error prevention closed loop.

10. An AI-powered global vision-based error-proofing system for flexible automotive production lines with multiple vehicle models on the same line, including: One or more processors; Memory, used to store one or more programs; Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 9.

Images