New energy automobile flexible production line AI fool-proof monitoring system and method based on multi-mode perception

Abstract

The invention provides a new energy automobile flexible production line AI fool-proof monitoring system and method based on multi-mode perception. Belongs to the technical field of intelligent manufacturing. The method comprises the following steps: carrying out key monitoring area division on a new energy automobile flexible production line, and generating monitoring area data of each station of the production line; according to monitoring area data of each station of a production line, intelligent monitoring equipment is deployed, and a multi-modal intelligent monitoring layout network is constructed; based on a multi-mode intelligent monitoring layout network, adaptive monitoring frequency adjustment is carried out according to different vehicle types, process parameters and production line real-time states; and obtaining multi-source original monitoring data and production line monitoring timestamp data. An intelligent monitoring layout network is constructed by deploying multi-mode sensors such as visual sensors, force sensors and laser radars in a new energy automobile flexible production line, production line operation information can be captured in an omnibearing and multi-dimensional mode, and the limitation of monitoring of a traditional single-mode sensor is broken through.

Description

TECHNICAL FIELD

[0001] The application provides a new energy automobile flexible production line AI foolproof monitoring system and method based on multi-modal perception, and belongs to the technical field of intelligent manufacturing. BACKGROUND

[0002] At present, flexible production lines have become the key to improving production efficiency and competitiveness in the new energy automobile industry due to their ability to quickly adapt to different vehicle models and process changes. However, the current foolproof measures for new energy automobile flexible production lines (including battery pack assembly, electric drive assembly, chassis integration, etc.) have many shortcomings.

[0003] Existing foolproof methods mainly rely on single-mode sensors, such as using only 2D cameras to detect the presence or absence of materials, or using only force sensors to determine whether tightening is in place. This single detection method is difficult to deal with complex errors, such as "correct parts installed in the wrong position", which is difficult to detect. At the same time, the threshold alarm method sets fixed upper and lower limits for torque, displacement, etc., which cannot adapt to dynamic working conditions. Different vehicle models, temperature changes, etc. will affect the accuracy of detection. Moreover, abnormalities are usually discovered at the final inspection station, and defective products have already flowed into subsequent processes, resulting in high repair costs. In addition, the rule engine based on IF-THEN logic is rigid and difficult to adapt to frequent model switching and process changes in flexible production lines. Some manufacturers claim "AI vision quality inspection" and "intelligent error prevention", but they simply stack multiple independent detection modules without modeling the cause-and-effect chain in a unified spatiotemporal coordinate system based on visual, force, and laser radar data, making it impossible to achieve autonomous evolution of the foolproof strategy. Therefore, it is urgent to develop an effective foolproof monitoring method. SUMMARY

[0004] The application provides a new energy automobile flexible production line AI foolproof monitoring system and method based on multi-modal perception to solve the problems mentioned in the background art:

[0005] The new energy automobile flexible production line AI foolproof monitoring method based on multi-modal perception provided by the application has the following characteristics:

[0006] S1, dividing the key monitoring area of the new energy automobile flexible production line to generate production line monitoring area data of each station; deploying intelligent monitoring equipment according to the production line monitoring area data of each station, and constructing a multi-modal intelligent monitoring layout network;

[0007] S2, based on the multi-modal intelligent monitoring layout network, adjusting the monitoring frequency according to different vehicle models, process parameters, and real-time state of the production line; obtaining multi-source original monitoring data and production line monitoring timestamp data; performing data preprocessing on the multi-source original monitoring data to generate high-quality multi-modal monitoring data;

[0008] S3, constructing a causal inference model under a unified space-time coordinate system according to high-quality multi-modal monitoring data; performing fusion analysis on the multi-modal monitoring data to generate abnormal comprehensive evaluation data;

[0009] S4, performing three-dimensional laser scanning processing on the new energy vehicle flexible production line through production line monitoring timestamp data to generate production line three-dimensional point cloud data; performing working condition activity analysis on the production line three-dimensional point cloud data in combination with the abnormal comprehensive evaluation data to generate abnormal working condition activity data; and predicting subsequent problems caused by the abnormality to obtain foolproof risk prediction data;

[0010] S5, performing weighted risk index processing according to the foolproof risk prediction data to generate a foolproof risk index; setting different risk levels according to the foolproof risk index, triggering a foolproof intervention mechanism when the risk index reaches a preset threshold, and generating new energy vehicle flexible production line AI foolproof early warning data.

[0011] The new energy vehicle flexible production line AI foolproof monitoring system based on multi-modal perception provided by the application comprises:

[0012] one or more processors;

[0013] a memory for storing one or more programs;

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of the above.

[0015] The application has the beneficial effects that: by deploying visual, force perception, laser radar and other multi-modal sensors in the flexible production line of new energy vehicles to construct an intelligent monitoring layout network, production line operation information can be captured in all directions and multidimensionally, the limitations of traditional single-mode sensor monitoring are broken, compound errors such as correct parts being installed in the wrong position are effectively identified, the probability of abnormal conditions such as misinstallation and missed installation caused by incomplete monitoring is greatly reduced, the precision of the foolproofing of the production line is improved, and more reliable protection is provided for product quality; a causal reasoning model is constructed based on a unified space-time reference, multi-source data is fused and analyzed, and real-time adaptation to different vehicle models, process parameters and dynamic working conditions is achieved, and the alarm and rigid rule engine are no longer dependent on fixed thresholds. This makes the foolproofing strategy be able to adjust independently according to the actual situation, flexibly cope with frequent vehicle model switching and process changes of the flexible production line, significantly enhance the flexibility and adaptability of the production line, and reduce misjudgment and missed judgment caused by working condition changes; by combining production line monitoring timestamp data with multi-source monitoring data, three-dimensional laser scanning and working condition activity analysis are performed, the subsequent problems that may be caused by abnormalities can be predicted in advance, and a foolproof intervention mechanism is triggered. This active safety protection system changes the passive situation of traditional post-interception, solves abnormal problems in the embryonic state, reduces the risk of defective products flowing into subsequent processes, reduces the cost of rework, improves production efficiency and economic benefits; a foolproof risk index is generated by weighted risk index processing and a risk level is set, precise risk warning and intervention are realized. The situation of excessive intervention or insufficient intervention is avoided, the blindness of manual intervention is reduced, the production line operation is more intelligent and efficient, and a new generation of active safety protection barrier is constructed for the flexible production line of new energy vehicles. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The method steps of the application are described. DETAILED DESCRIPTION

[0017] The preferred embodiments of the application are described below in conjunction with the accompanying drawings, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the application, and are not used to limit the application.

[0018] An embodiment of the application is shown in Figure 1 A multi-modal perception-based AI foolproof monitoring method for a flexible production line of new energy vehicles, the method comprising:

[0019] S1, dividing the key monitoring areas of the flexible production line of new energy vehicles (including battery pack assembly, electric drive assembly, chassis integration and other stations), generating production line station monitoring area data; deploying visual sensors, force perception sensors, laser radar sensors and other intelligent monitoring devices according to the production line station monitoring area data, constructing a multi-modal intelligent monitoring layout network, and ensuring that the sensors work cooperatively under a unified space-time reference;

[0020] S2, based on the multi-modal intelligent monitoring layout network, according to different vehicle models, process parameters and real-time state of the production line, the adaptive monitoring frequency is adjusted; at the same time, the image data collected by the visual sensor, the torque and pressure data collected by the force sensor, and the spatial position data collected by the laser radar sensor are used to obtain multi-source original monitoring data and production line monitoring timestamp data; the multi-source original monitoring data is preprocessed, including image denoising, force data filtering, laser radar point cloud denoising, etc., to generate high-quality multi-modal monitoring data;

[0021] S3, according to the high-quality multi-modal monitoring data, a causal reasoning model is constructed in a unified space-time coordinate system; the model is used to fuse and analyze the multi-modal monitoring data, to identify abnormal conditions such as part misassembly, torque overrun, path deviation, etc. in real time, to obtain part assembly state data, torque state data, and path deviation state data; through causal correlation analysis between the part assembly state data, the torque state data, and the path deviation state data, the abnormal conditions are comprehensively evaluated to generate abnormal comprehensive evaluation data;

[0022] S4, the new energy vehicle flexible production line is processed by three-dimensional laser scanning through the production line monitoring timestamp data, to generate production line three-dimensional point cloud data; combined with the abnormal comprehensive evaluation data, the production line three-dimensional point cloud data is analyzed for working condition activity, to identify the position, range and possible influence area of the abnormal occurrence, to generate abnormal working condition activity data; based on the abnormal working condition activity data, a pre-trained risk prediction model is used to predict the subsequent problems (such as part damage, production line downtime, etc.) that may be caused by the abnormality, to obtain foolproof risk prediction data;

[0023] S5, according to the foolproof risk prediction data, combined with different vehicle models, process requirements and historical data, weighted risk index processing is performed to generate a foolproof risk index; according to the foolproof risk index, different risk levels are set, when the risk index reaches a preset threshold, a foolproof intervention mechanism is triggered, including automatic shutdown, alarm prompt, process parameter adjustment, etc., to generate new energy vehicle flexible production line AI foolproof warning data, to realize active safety protection of the production line.

[0024] The working principle and effects of the above technical solutions are: through the layout and cooperative work of the multi-modal sensor, the accuracy of the line abnormality identification is improved, and problems such as part misloading and torque overrun can be captured in time. The adaptive monitoring frequency adjustment and data preprocessing make the monitoring more efficient and avoid resource waste. The causal reasoning model analyzes the multi-modal data fusion, enhances the reliability of abnormality evaluation, and reduces the false intervention caused by single data misjudgment. Combined with the time stamp, the three-dimensional scanning and working condition analysis quickly locate the abnormal position and influence range, greatly shorten the problem troubleshooting time. The risk prediction and active intervention mechanism avoids subsequent problems such as part damage and line downtime, reduces the rework cost and production loss. At the same time, it adapts to different vehicle models and process requirements, not only meets the flexible demand of flexible production, but also builds a safety protection line, and improves the stability of the production line.

[0025] In one embodiment of the present application, the S1 comprises:

[0026] S11, the process flow is disassembled and precision requirement analysis is performed on the core workstations (battery pack assembly, electric drive assembly, chassis integration, etc.) of the new energy vehicle flexible production line, the key operation nodes (such as battery pack bolt tightening, electric drive assembly joint angle) and potential risk points (such as part mistake-proofing identification, assembly force threshold) of each workstation are determined, and workstation characteristic investigation data is generated;

[0027] S12, based on the workstation characteristic investigation data, combined with the spatial layout and operation range of each workstation (such as a 3m×2m operation area of the battery pack assembly workstation and a 5m×3m docking area of the chassis integration workstation), a spatial gridding partition method is used to divide the monitoring area of each workstation, so as to ensure that there is no monitoring blind area, and workstation monitoring area data is generated;

[0028] S13, based on the workstation monitoring area data, according to the monitoring requirements of different areas (such as visual identification of part positioning for battery pack assembly, force detection of torque for electric drive assembly, and laser radar capture of space path for chassis integration), the appropriate intelligent monitoring equipment (resolution≥2K visual sensor, range 0-500N·m force sensor, ranging accuracy±2mm laser radar sensor) is selected, and monitoring equipment selection matching data is generated;

[0029] S14, based on the monitoring equipment selection matching data, the corresponding intelligent monitoring equipment is deployed in each monitoring area; through GPS synchronous clock and spatial coordinate calibration technology, the time stamps of all sensors are unified to the line central clock system, and the spatial coordinates are unified to the line reference coordinate system (with the chassis integration workstation origin as (0, 0, 0)), so as to eliminate the space-time deviation, and generate sensor deployment and space-time calibration data;

[0030] S15, utilize sensor deployment and space-time calibration data, construct the communication link between devices (adopt 5G+ edge computing to realize low delay transmission), set the cooperative trigger rule of each sensor (such as visual sensor identifies that part is in place, and then force sensor is started to collect torque), and generate multi-modal intelligent monitoring layout network.

[0031] The working principle and effect of the above technical scheme are: by disassembling the core station process and analyzing the precision demand, finding out the key nodes and risk points of each link, combining with the space layout to do grid partition, the monitoring blind area is completely filled, which greatly improves the accuracy of monitoring coverage. According to the needs of different areas, appropriate equipment is selected, such as high-definition visual sensor for battery pack assembly, force sensor for electric drive assembly, the collected data is more in line with the actual demand, and the reliability of monitoring is enhanced. Through clock synchronization and coordinate calibration, the time and space reference of all sensors are kept consistent, which avoids misjudgment caused by time and space deviation during subsequent data fusion, and reduces invalid data interference. The communication link of 5G+ edge computing is matched with the cooperative trigger rule, such as automatically starting torque collection after the part is in place, so that the equipment response is more timely, and the data transmission delay is reduced. This layout can accurately match the differentiated monitoring needs of each station, and can ensure smooth cooperation of multiple devices, which lays a solid foundation for subsequent abnormal identification and risk prevention and control.

[0032] In an embodiment of the present application, the S15 comprises:

[0033] Based on sensor deployment and space-time calibration data, the physical location and data transmission demand (such as visual image needs high bandwidth, force torque data needs low delay) of each intelligent monitoring device (visual sensor, force sensor, laser radar) are determined; 5G+ edge computing layered communication architecture is constructed: edge layer deploys localized edge gateway (1 per station, supports multiple device access), network layer adopts 5G private network (slice bandwidth≥100Mbps, time delay≤10ms), core layer connects the central control system of the production line; At the same time, the communication interface type (such as Ethernet / IP, RS485) and data transmission priority (force data>laser radar data>visual image data) of each device are labeled, and a monitoring device communication architecture planning scheme is generated;

[0034] According to the monitoring device communication architecture planning scheme, edge gateway and 5G communication module are deployed at each station, and intelligent monitoring devices and edge gateway are connected through network cable or wireless AP; the data transmission protocol between devices is configured (such as Profinet protocol for real-time torque data transmission of force sensor, HTTP / 2 protocol for compressed image transmission of visual sensor), and test data packets are sent through the production line central control system to verify the communication connectivity of each device and edge gateway, edge gateway and core system (packets loss rate ≤0.1% is qualified); after excluding communication interruption, data loss and other problems, multi-device communication link establishment completion data is generated;

[0035] Based on the multi-device communication link establishment completion data, the linkage logic of each station sensor is sorted out: for example, after the visual sensor recognizes the part positioning completion signal, the force sensor needs to be triggered to start torque collection at the battery pack assembly station; at the electric drive assembly station, the laser radar needs to trigger the visual sensor to confirm the docking angle again after detecting that the assembly has reached the docking position; the trigger conditions (such as visual recognition success rate ≥95%), trigger delay (≤50ms) and abnormal backup mechanism (when the main sensor fails, the standby sensor is automatically triggered) are determined, and sensor cooperative trigger rule design data is generated;

[0036] The sensor cooperative trigger rule design data is imported into the rule engine of the production line central control system, and the actual production scene of each station (such as battery pack part misplacement, electric drive assembly angle deviation) is simulated to test the sensor linkage response: the visual trigger force delay, laser radar trigger visual accuracy rate are recorded, and the problems of false trigger (such as force trigger without parts) or missed trigger (such as force not starting after parts are in place) are investigated; the rule parameters are optimized (such as adjusting the confidence threshold of visual recognition) for abnormal conditions until the trigger success rate ≥99.5%, and cooperative trigger rule verification passed data is generated;

[0037] Based on the multi-device communication link establishment completion data and the cooperative trigger rule verification passed data, the intelligent monitoring devices, communication links and cooperative rules of each station are integrated into a unified system: the device online state, communication link stability and trigger rule execution are visualized on the central control system interface; at the same time, data real-time feedback and abnormal alarm functions (automatic pop-up alarm when communication is interrupted, log recording when trigger fails) are configured to ensure that the entire monitoring system is manageable, controllable and traceable, and a multi-modal intelligent monitoring layout network is generated.

[0038] The working principle and effects of the above technical solution are that: the hierarchical architecture of 5G+ edge computing is matched with the communication priority setting of the equipment, so that the high bandwidth demand of visual images and the low delay demand of force sense data can be considered, and the data transmission delay is greatly reduced. Interface adaptation and connectivity testing exclude problems such as communication interruption and data packet loss in advance, reducing the data loss caused by communication failure in subsequent monitoring. The cooperative triggering rule of the fitting station logic, such as triggering force sensing collection after visual confirmation of part positioning, plus the main and backup sensor backup mechanism, avoids the situation of false triggering or missed triggering, and enhances the accuracy of equipment linkage. The visual interface and abnormal alarm function after system integration make the equipment state and link stability visible in real time, and can alarm in time when communication interruption or triggering fails, so that the running state of the whole system is clear and controllable, and the problem can be quickly checked. This design can not only meet the differentiated transmission needs of different equipment, but also ensure smooth cooperation of multiple equipment, providing stable bottom support for accurate production line monitoring.

[0039] In one embodiment of the present application, the S2 comprises:

[0040] S21, collect information such as current production models (such as pure electric cars, hybrid SUVs), process parameters (such as battery pack bolt tightening torque standard 80±5N·m, electric drive assembly beat 60s / table), and production load (such as the number of work-in-process at the current station) through the production line MES system (manufacturing execution system), and generate real-time working condition data of the production line;

[0041] S22, based on the multi-modal intelligent monitoring layout network and the real-time working condition data of the production line, a frequency adjustment model is established: when the production beat is improved by 20%, the visual sensor acquisition frequency is improved from 10fps to 15fps; when the process parameter requires higher accuracy, the force sensing sensor sampling frequency is improved from 100Hz to 200Hz, and adaptive monitoring frequency parameters are generated.

[0042] S23, based on the generated adaptive monitoring frequency parameters, control the synchronous data acquisition of each sensor, including: visual sensor acquires part assembly image, force sensing sensor acquires bolt tightening torque and part pressing pressure, laser radar sensor acquires spatial position coordinates of assembly parts, and records the production line monitoring time stamp corresponding to each group of data at the same time, generates multi-source original monitoring data+production line monitoring time stamp data;

[0043] S24, type classification preprocessing is performed on the multi-source original monitoring data, and the classification preprocessing includes removing noise from image data using a Gaussian filtering algorithm (eliminating workshop light interference), smoothing force sensing data using a Kalman filtering algorithm (filtering mechanical vibration error), and removing outliers from laser radar point cloud data using a statistical filtering algorithm (eliminating invalid points generated by dust shielding), to generate single-mode preprocessed data;

[0044] S25, based on the production line monitoring timestamp data, the single mode pretreatment data is time and space alignment (ensure that the image, force sense, point cloud data corresponding to the same assembly action under the same timestamp), and the data consistency is checked (such as the deviation of the part position collected by the laser radar and the part position identified by the image should be less than or equal to 1mm), and high-quality multi-modal monitoring data is generated.

[0045] The working principle and effect of the above technical solution are: the monitoring frequency is adjusted combined with the working condition data of the MES system, when the production rhythm is accelerated or the precision requirement is improved, the collection frequency is correspondingly improved, so that the monitoring precision matches the production rhythm- neither missing the key action due to low frequency, nor wasting resources caused by high frequency collection, and the adaptive flexibility of monitoring is improved. The type-specific pretreatment is very targeted, the image uses Gaussian filtering to eliminate light interference, the force sense data uses Kalman filtering to smooth vibration, and the laser point cloud removes outliers, so that the impurities of the processed data are greatly reduced, the reliability of subsequent analysis is enhanced, and the misjudgment caused by noise is reduced. The time stamp is used to realize the time and space alignment of multi-modal data, and the millimeter-level consistency verification is also performed, so that the data of different sensors accurately correspond to the same assembly action, and the error evaluation caused by data misplacement is avoided. Through the whole process, it can flexibly adapt to the dynamic changes of different vehicle models and processes, and can stably output high-quality monitoring data, providing a solid foundation support for subsequent abnormal identification and risk prediction.

[0046] In an embodiment of the present application, the S25 comprises:

[0047] Based on the production line monitoring timestamp data, the original timestamp of the single mode pretreatment data (image data, force sense data, laser radar point cloud data) is extracted one by one; the original timestamp is compared with the standard time of the production line central clock system, and the time difference of each modal data is eliminated (the deviation is less than or equal to 1ms) through a time offset compensation algorithm (such as linear interpolation correction) to generate single mode pretreatment data with unified timestamp;

[0048] Based on the single mode pretreatment data with unified timestamp, the determined production line reference coordinate system parameters (with the chassis integrated station origin as (0, 0, 0)) are called; the image pixel coordinates of the visual sensor (converted through the camera intrinsic matrix), the installation position coordinates of the force sense sensor, and the point cloud absolute coordinates of the laser radar are uniformly mapped to the reference coordinate system to eliminate the spatial dimension deviation, and the time and space related single modal data set is generated;

[0049] Based on the spatiotemporal correlation of the single-mode data set, combined with the production line assembly process library (such as the standard action library of battery pack bolt tightening and electric drive assembly docking), the multi-modal data corresponding to the same assembly action is matched according to the timestamp sequence: for example, the bolt torque data (force sense), bolt head image (vision), and bolt spatial position (laser radar) at the 08:30:05 timestamp are grouped together to generate a multi-modal data action correlation group;

[0050] Based on the multi-modal data action correlation group, set the key parameter consistency check rule: the deviation of the part spatial position collected by the laser radar and the part position recognized by the vision image needs to be ≤1mm, and the deviation of the assembly force collected by the force sensor and the process standard force value needs to be ≤5%; cross-check each group of data, mark the abnormal data group with deviation exceeding the standard, and generate a multi-modal data group after consistency check;

[0051] Based on the multi-modal data group after consistency check, perform hierarchical processing on the marked abnormal data group: slight deviation (such as position deviation 0.8-1mm) is corrected by data fusion algorithm (such as weighted average), and serious deviation (such as position deviation >1mm) triggers sensor resampling instruction; after the abnormal processing is completed, integrate all qualified data groups to generate high-quality multi-modal monitoring data.

[0052] The working principle and effect of the above technical solution are: through timestamp comparison and linear interpolation correction, the time difference of each modal data is controlled within 1ms, completely solving the problem of different sensor data asynchronization, and improving the time correlation of the data. Image pixels, force sense installation position, and laser point cloud coordinates are mapped to the production line reference coordinate system, eliminating the spatial dimension deviation, so that data from different sources can accurately correspond to the same assembly scene. According to the timestamp, the multi-modal data of the same assembly action is matched, such as grouping the bolt torque, head image, and spatial position at the same time, and then cross-checking through the verification rule of millimeter-level position deviation and force value deviation, reducing the interference of invalid data. For slight deviation, the algorithm is corrected, and for serious deviation, resampling is triggered, avoiding the inflow of inferior data into the subsequent link, and greatly improving the quality of the final monitoring data. After such processing, the spatiotemporal consistency of multi-modal data is ensured, and the reliability of the data is ensured through verification and correction, providing solid data support for subsequent accurate identification of part misinstallation, torque overrun, and other abnormalities.

[0053] One embodiment of the present application, the S3, comprises:

[0054] S31, based on high-quality multi-modal monitoring data, analyze the correlation logic of different types of data, such as the correlation between part positioning deviation in visual data and spatial path deviation in laser radar data, the correlation between torque overrun in force perception data and bolt model misloading in visual data, and the correlation between torque overrun in force perception data and bolt model misloading in visual data, and determine the correlation dimension (position, force, time), and generate multi-modal data correlation dimension definition data;

[0055] S32, combine multi-modal data correlation dimension definition data with production line historical abnormal case library (including part misloading, torque overrun, path deviation cases in the past 3 years and corresponding processing results), and train causal inference model using Bayesian network algorithm, learn the mapping relationship between abnormal phenomenon and root cause (such as torque overrun may be caused by bolt model error or assembly angle deviation), and generate pre-trained causal inference model;

[0056] S33, input high-quality multi-modal monitoring data into the pre-trained causal inference model, and identify abnormalities by dimension: identify part misloading (such as battery pack positive and negative terminal connection) through visual data, generate part assembly state data; identify torque overrun (such as bolt tightening torque 95N·m exceeding the standard range) through force perception data, generate torque state data; identify path deviation (such as chassis conveying track deviation 3mm) through laser radar data, generate path deviation state data;

[0057] S34, extract abnormal features (such as misloaded part model, overrun torque value, and offset path distance) from part assembly state data, torque state data, and path deviation state data, analyze the causal relationship between abnormalities (such as part misloading leading to torque overrun, and path deviation aggravating part assembly deviation) based on the pre-trained causal inference model, and generate abnormal causal correlation data;

[0058] S35, combine abnormal causal correlation data, set evaluation indicators (abnormal influence range, occurrence probability, rectification difficulty), calculate the comprehensive influence weight of each abnormality (such as part misloading weight 0.4, torque overrun weight 0.3, path deviation weight 0.3) using AHP method, and generate abnormal comprehensive evaluation data.

[0059] The working principle and effects of the above technical solution are as follows: first, the correlation logic of different modal data is clarified, such as the corresponding relationship between part positioning deviation and space path offset, then combined with three years of historical abnormal cases, a causal reasoning model is trained by using a Bayesian network, so that the model can find out the mapping rule between the abnormality and the root cause, such as accurately judging that torque overrun may be caused by incorrect installation of bolt type, rather than simply mechanical problems, which greatly improves the accuracy of abnormal identification and avoids misjudgment of single-dimensional judgment. After identifying part misinstallation, torque overrun and other abnormalities in different dimensions, the model is used to analyze the causal relationship between each other, such as finding that part misinstallation will cause torque overrun, and path offset will aggravate assembly deviation, which reduces the invalid rectification of only treating surface problems and ignoring the root cause. The importance of different abnormalities is obvious by using the analytic hierarchy process to evaluate the impact range, rectification difficulty and other indicators, which enhances the rationality of evaluation. The whole process can comprehensively capture various abnormalities and dig the correlation and root cause between abnormalities, which provides a reliable basis for subsequent accurate treatment scheme.

[0060] In one embodiment of the present application, the S4 comprises:

[0061] S41, based on the production line monitoring timestamp data, screening the key time nodes (such as 10s before the abnormal alarm triggering to 5s after the triggering) of the abnormal occurrence period, setting the time interval (0.5s / time) and scanning range (covering the abnormal workstations and adjacent 2 workstations) of the three-dimensional laser scanning, generating three-dimensional laser scanning time sequence control data;

[0062] S42, according to the three-dimensional laser scanning time sequence control data, starting the laser radar scanning equipment to dynamically scan the new energy automobile flexible production line, obtaining the production line space point cloud information of each time node, fusing multiple frames of point cloud data through a point cloud splicing algorithm (such as ICP algorithm), and generating production line three-dimensional point cloud data;

[0063] S43, spatially matching the abnormal comprehensive evaluation data and the production line three-dimensional point cloud data, labeling the point cloud area corresponding to the abnormality (such as the battery pack point cloud area corresponding to part misinstallation, and the bolt point cloud area corresponding to torque overrun), combining the production line CAD model to determine the specific position (such as X=2.5m, Y=1.8m, Z=0.6m) and influence range (diameter 0.3m) of the abnormality, and generating abnormal working condition activity data;

[0064] S44, extracting the key features (abnormal position, influence range, duration) in the abnormal working condition activity data, combining the production line equipment parameters (such as bolt material strength, conveying track bearing) and process requirements (such as assembly accuracy ±0.5mm), and arranging into an input feature vector of a risk prediction model to generate risk prediction input data;

[0065] S45, input the risk prediction input data into the pre-trained deep learning risk prediction model (based on ResNet architecture, the training set contains 100,000+ abnormal cases), predict the subsequent problems that may be caused by the abnormality (such as the probability of circuit short circuit caused by part misloading, the probability of bolt fracture caused by torque overrun, and the probability of equipment collision caused by path deviation), and generate foolproof risk prediction data.

[0066] The working principle and effect of the above technical solution are: screening the key time period before and after the abnormality occurs, focusing on the abnormal station and adjacent area scanning, and still controlling the scanning interval of 0.5 seconds / time, which not only accurately captures the abnormal process, but also avoids the waste of resources caused by full-time and large-scale scanning, and improves the scanning efficiency. By splicing multiple frames of point cloud to generate three-dimensional data through the ICP algorithm, and combining with the abnormal evaluation data to label the point cloud area, and locking the specific coordinates and influence range with the CAD model of the production line, such as accurately positioning the bolt abnormality at (2.5, 1.8, 0.6) position and affecting 0.3 meter range, the time cost of abnormality investigation is greatly reduced, and the blindness of finding problems by experience is avoided. After extracting the abnormal features, inputting the device parameters and process requirements into the model, this model trained based on 100,000+ cases can accurately predict the subsequent risks, such as the probability of short circuit caused by part misloading and the probability of bolt fracture caused by torque overrun, which enhances the reliability of risk prediction. In this way, the core position of the abnormality can be quickly locked, and the subsequent hazards can be found out in advance, which provides strong support for timely intervention measures to avoid production line downtime or part damage.

[0067] In an embodiment of the present application, the S42 comprises:

[0068] S421, according to the three-dimensional laser scanning time sequence control data, extract the key control parameters: scanning time interval (0.5s / time), scanning range (space coordinate range of abnormal station and adjacent 2 stations), scanning accuracy (point cloud density ≥100 points / cm²); import these parameters into the control module of the laser radar scanning equipment, synchronize the scanning angle (horizontal scanning range 0-360°, vertical scanning range -15° to 90°) of the equipment with the data sampling frequency, ensure that the equipment parameters and control data are completely matched, and generate laser radar equipment start-up ready data.

[0069] S422, based on the laser radar equipment start-up ready data, trigger the equipment to start dynamic scanning according to the set time sequence: at the key time nodes of the abnormal occurrence period (such as 10s before alarm, 5s before alarm, alarm time, 2s after alarm, 5s after alarm), collect the production line space point cloud information at the corresponding time; each frame of point cloud data is associated with a corresponding time stamp and a scanning position label (such as station A-alarm time-08:45:30), to avoid data confusion, and generate a multi-time node original point cloud data set;

[0070] S423, based on the multi-time node original point cloud data set, performing preprocessing operation on each frame of point cloud: adopting statistical filtering algorithm to remove outlier points (keeping point cloud data with confidence degree greater than or equal to 99%) generated by dust and light interference, adopting voxel grid downsampling algorithm to compress data amount (reducing calculation load under the premise of ensuring accuracy), and adopting radius filtering algorithm to remove isolated points (filtering single-point noise); after processing, ensuring that the point cloud data is free of invalid information, and generating clean single-frame point cloud data;

[0071] S424, based on the clean single-frame point cloud data, calling ICP (iterative closest point) splicing algorithm: taking point cloud data at the alarm time as a reference frame, performing feature point matching (extracting point cloud edges, corner points and other features) between the reference frame and point cloud frames at other time nodes, and minimizing frame distance error (ensuring that the splicing error is less than or equal to 0.5 mm) through iterative calculation; gradually fusing point cloud data at all time nodes, forming a continuous spatial point cloud model covering the entire abnormal period, and generating preliminary spliced point cloud data.

[0072] S425, based on the preliminary spliced point cloud data, checking quality through two dimensions: spatial continuity (checking whether there is a fault at the junction of adjacent frame point clouds, and the fault gap needs to be less than or equal to 0.3 mm), and time consistency (verifying whether the point cloud corresponding to each time node matches the actual process action); if there is a quality problem (such as a fault or misplacement), returning to S424 to re-adjust the iteration number of the ICP algorithm; after passing the verification, finally generating production line three-dimensional point cloud data.

[0073] The working principle and effect of the above technical solution are as follows: first, the scanning interval, range and accuracy of the laser radar are calibrated according to the control data, and the scanning angle and sampling frequency are also calibrated, so that the device parameters completely fit the requirements, avoiding point cloud data distortion caused by parameter deviation and improving the reliability of the original data. According to the key node time sequence of the abnormal occurrence, each frame of data is associated with a time stamp and a station label, which not only accurately covers the complete process before and after the alarm, but also completely eliminates the problem of data confusion. The multi-round preprocessing is very accurate, the statistical filtering removes outliers, the voxel downsampling compresses data, and the radius filtering removes isolated points, so that the processed data is clean and noise-free, and the calculation load of subsequent splicing is also reduced. Using the ICP algorithm to splice the point cloud at the alarm time as a reference, the feature points are accurately matched, the frame error is pressed to within 0.5 mm, and then the spatial continuity and time consistency are checked to ensure that there is no fault or misplacement after splicing, and it is completely matched with the actual production line state. The three-dimensional point cloud data generated in this way can clearly restore the production line space state during the entire abnormal period, and also ensures the data accuracy and reliability, providing high-quality core data support for subsequent positioning of abnormal positions and analysis of influence range.

[0074] In an embodiment of the present application, the S424 comprises:

[0075] Based on the cleaned single-frame point cloud data, the alarm time point cloud frame of the abnormal occurrence period is screened as the splicing reference frame; the reference frame is executed again for fine feature enhancement (the edge features of the point cloud are enhanced by the Sobel operator, and the key structure corner points are highlighted by the Harris corner detection algorithm), to ensure that the features of the reference frame are clear and identifiable, and to generate reference frame enhanced point cloud data.

[0076] Based on the reference frame enhanced point cloud data, the same feature extraction operation is performed on the remaining cleaned single-frame point cloud (10s before the alarm, 5s before the alarm, 2s after the alarm, and 5s after the alarm) with reference to its feature types (edge and corner); each non-reference frame generates a set containing edge points, corner point coordinates and feature descriptors, to avoid splicing deviation caused by mismatched feature types, and to generate a multi-frame point cloud feature point set.

[0077] The initial matching module of the ICP algorithm is called to compare each frame of feature points in the multi-frame point cloud feature point set with the feature points of the reference frame enhanced point cloud data one by one; the matching pairs are screened by calculating the feature descriptor similarity (similarity threshold ≥ 0.9), and the false matching points (such as isolated noise points) are excluded, to generate the initial matching result of the inter-frame feature points.

[0078] Based on the initial matching result of the inter-frame feature points, the point cloud distance error (such as the mean of the Euclidean distance) between the current frame and the reference frame is calculated; if the error is > 0.5mm, the ICP algorithm iteration optimization is started: the spatial pose (translation, rotation) of the non-reference frame is adjusted, the distance of the matching point pairs is recalculated, until the error is ≤ 0.5mm and stable convergence is achieved, to generate the inter-frame optimized matching data.

[0079] Based on the inter-frame optimized matching data of all frames, the non-reference frame point cloud is gradually fused into the reference frame coordinate system in time sequence: first, the pre-alarm frames (from 10s to 5s) are fused, and then the post-alarm frames (from 2s to 5s) are fused, while maintaining the time continuity of the point cloud data; after the fusion is completed, a spatial point cloud model covering the entire abnormal period is formed, and the preliminary spliced point cloud data is generated.

[0080] The working principle and effects of the above technical solution are as follows: the point cloud frame of the alarm time is selected as the reference frame, the edge is strengthened by using the Sobel operator, and the corner point is highlighted by using the Harris algorithm, so that the key features of the reference frame are clearly visible, and the stable reference is provided during splicing, thereby avoiding the overall misplacement caused by the reference blur. The features of the non-reference frame are extracted according to the standard of the reference frame, so that the edge and the corner point type are consistent, and the splicing deviation caused by the feature mismatch is reduced from the root. When the ICP algorithm is used for matching, the matching pairs are first screened with a similarity of 0.9, the false matching of the isolated noise points is eliminated, and then the point cloud distance error is optimized to be within 0.5 mm, so that the splicing accuracy is greatly improved. The point clouds are gradually fused in the time sequence before and after the alarm, so that the time continuity of the data is ensured, and the whole time period of the abnormality is completely covered. The point cloud model spliced in this way can accurately restore the space state of the production line during the abnormal process, and clearly retains the details of each time node, thereby providing high-quality three-dimensional data support for subsequent accurate positioning of the abnormal position and analysis of the influence range.

[0081] In one embodiment of the present application, the S5 comprises:

[0082] S51, extract the foolproof risk handling records in the past 5 years from the production line database, count the historical losses of different abnormal types (such as part damage cost, production line downtime length, rework man-hours), establish a historical risk loss weight table (such as part damage weight 0.3, downtime loss weight 0.5, rework weight 0.2), and generate historical risk reference data;

[0083] S52, combine the foolproof risk prediction data (such as risk occurrence probability, estimated loss) with the historical risk reference data, set a weighted formula: foolproof risk index = (risk occurrence probability x 0.6) + (historical risk loss weight of the same type x 0.4), substitute the data to obtain a specific value (such as 0-100 points), and generate a foolproof risk index;

[0084] S53, set risk level standards according to the foolproof risk index: 1-30 points for grade I (low risk), 31-60 points for grade II (medium risk), and 61-100 points for grade III (high risk), set corresponding response thresholds for each grade (such as grade II risk threshold 31 points, grade III risk threshold 61 points), and generate risk level division standards;

[0085] S54, compare the foolproof risk index with the risk level division standards: if it is grade I risk, trigger an alarm prompt (workshop sound and light alarm); if it is grade II risk, trigger process parameter adjustment (such as automatically correcting the bolt tightening torque); if it is grade III risk, trigger automatic shutdown (emergency power-off of the production line), and generate a foolproof intervention trigger instruction;

[0086] S55, based on the fool-proof intervention trigger instruction, record relevant information, the relevant information includes intervention measure type, trigger time, involved station and processing person in charge, generate new energy automobile flexible production line AI fool-proof early warning data; at the same time, the early warning data is synchronized to the production line MES system and the management personnel terminal, guiding the on-site personnel to execute subsequent processing (such as part replacement, equipment calibration), realizing active safety protection.

[0087] The working principle and effect of the above technical solution are as follows: according to the five-year fool-proof risk processing record, the historical loss weight is calculated according to part damage, downtime loss, etc., so that the risk assessment has a solid historical basis, and the one-sidedness of subjective judgment weight is avoided. The index is calculated by combining the risk occurrence probability and the historical weight, the probability accounts for 60%, and the historical weight accounts for 40%, the calculated result is more in line with the actual production risk, and the accuracy of the evaluation is improved. Three levels of risk correspond to different interventions: low risk sound and light alarm, medium risk automatic adjustment of bolt torque and other process parameters, and high risk emergency shutdown, which neither over-intervenes due to small problems to affect the production rhythm, nor delays the treatment of high risks to cause accidents, reducing invalid downtime and part damage and other losses. The intervention information is synchronized to the MES system and the management personnel terminal, and the measure type, the person in charge and other details are also recorded in detail, which is convenient for subsequent tracing and optimization, so that the protection is not only passive response, but also forms a closed loop of early warning-intervention-recording. In this way, the risk level can be matched accurately to give a response scheme, and active protection can be realized, greatly improving the safety of the production line operation.

[0088] One embodiment of the present application is a new energy automobile flexible production line AI fool-proof monitoring system based on multi-modal perception, which comprises:

[0089] One or more processors;

[0090] Memory for storing one or more programs;

[0091] When the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of the above.

[0092] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application also intends to include these modifications and variations.

Claims

1. A multimodal perception-based AI-based error-proofing monitoring method for flexible production lines of new energy vehicles, characterized in that, The method includes: S1. Divide the key monitoring areas of the flexible production line for new energy vehicles and generate monitoring area data for each workstation of the production line; based on the monitoring area data of each workstation of the production line, deploy intelligent monitoring equipment and build a multimodal intelligent monitoring layout network. S2. Based on a multimodal intelligent monitoring network, the monitoring frequency is adjusted adaptively according to different vehicle models, process parameters, and real-time production line status; multi-source raw monitoring data and production line monitoring timestamp data are acquired; and the multi-source raw monitoring data is preprocessed to generate high-quality multimodal monitoring data. S3. Based on high-quality multimodal monitoring data, construct a causal reasoning model under a unified spatiotemporal coordinate system; perform fusion analysis on the multimodal monitoring data to generate comprehensive anomaly assessment data; S4. Perform 3D laser scanning processing on the flexible production line of new energy vehicles using production line monitoring timestamp data to generate 3D point cloud data of the production line; combine with anomaly comprehensive assessment data to perform working condition activity analysis on the 3D point cloud data of the production line to generate abnormal working condition activity data; and predict subsequent problems caused by anomalies to obtain error prevention risk prediction data. S5. Based on the error prevention risk prediction data, perform weighted risk index processing to generate an error prevention risk index; set different risk levels according to the error prevention risk index; when the risk index reaches the preset threshold, trigger the error prevention intervention mechanism to generate AI error prevention early warning data for the flexible production line of new energy vehicles.

2. The AI-based error-proofing monitoring method for flexible production lines of new energy vehicles based on multimodal perception as described in claim 1, characterized in that, S1 includes: S11. Decompose the process flow and analyze the precision requirements of the core workstations of the flexible production line for new energy vehicles, determine the key operation nodes and potential risk points of each workstation, and generate survey data on the characteristics of the production line workstations. S12. Based on the survey data of production line workstation characteristics, combined with the spatial layout and operation range of each workstation, the monitoring area of ​​each workstation is divided using the spatial grid partitioning method, and the monitoring area data of each workstation of the production line is generated. S13. Based on the monitoring data of each workstation on the production line, select suitable intelligent monitoring devices according to the monitoring needs of different areas, and generate monitoring device selection and matching data. S14. Based on the monitoring equipment selection and matching data, deploy corresponding intelligent monitoring equipment in each monitoring area; through GPS synchronization clock and spatial coordinate calibration technology, unify the timestamps of all sensors to the production line central clock system, unify the spatial coordinates to the production line reference coordinate system, and generate sensor deployment and spatiotemporal calibration data. S15. Utilize sensor deployment and spatiotemporal calibration data to construct communication links between devices, set collaborative triggering rules for each sensor, and generate a multimodal intelligent monitoring layout network.

3. The AI-based error-proofing monitoring method for flexible production lines of new energy vehicles based on multimodal perception as described in claim 2, characterized in that, S15 includes: Based on sensor deployment and spatiotemporal calibration data, the physical location and data transmission requirements of each intelligent monitoring device are determined; a 5G+edge computing hierarchical communication architecture is constructed, and a communication architecture planning scheme for monitoring devices is generated. Based on the communication architecture planning scheme of the monitoring equipment, edge gateways and 5G communication modules are deployed at each workstation. The intelligent monitoring equipment and the edge gateway are connected through network cables or wireless APs. The data transmission protocol between the devices is configured, and test data packets are sent through the production line central control system to verify the communication connectivity between each device and the edge gateway, and between the edge gateway and the core system, and to generate data showing the completion of the multi-device communication link construction. Based on the data established by the multi-device communication link, the linkage logic of sensors at each workstation is sorted out, and sensor collaborative triggering rule design data is generated. The sensor collaborative triggering rule design data is imported into the rule engine of the production line central control system to simulate the actual production scenarios of each workstation; the rule parameters are optimized for abnormal situations until the triggering success rate is ≥99.5%, and the collaborative triggering rule verification data is generated. Based on the completion of data and collaborative triggering rule verification through the construction of multi-device communication links, the intelligent monitoring devices, communication links, and collaborative rules of each workstation are integrated into a unified system to generate a multimodal intelligent monitoring layout network.

4. The AI-based error-proofing monitoring method for flexible production lines of new energy vehicles based on multimodal perception according to claim 1, characterized in that, The S2 includes: S21. Collect information on the current production vehicle model, process parameters, and production load through the production line MES system to generate real-time production line operating data. S22. Based on the multimodal intelligent monitoring layout network and real-time production line operating data, establish a frequency adjustment model and generate adaptive monitoring frequency parameters. S23. Based on the generated adaptive monitoring frequency parameters, control each sensor to collect data synchronously, and record the production line monitoring timestamp corresponding to each set of data to generate multi-source raw monitoring data + production line monitoring timestamp data. S24. Perform categorized preprocessing on the multi-source raw monitoring data to generate single-modal preprocessed data; S25. Based on the production line monitoring timestamp data, the single-modal preprocessed data is spatiotemporally aligned and the data consistency is verified to generate high-quality multimodal monitoring data.

5. The AI-based error-proofing monitoring method for flexible production lines of new energy vehicles based on multimodal perception according to claim 1, characterized in that, The S3 includes: S31. Based on high-quality multimodal monitoring data, analyze the correlation logic of different types of data and generate multimodal data correlation dimension definition data; S32. Combine multimodal data with the dimensions of the data association and the production line historical anomaly case library, use Bayesian network algorithm to train the causal inference model, and generate a pre-trained causal inference model. S33. Input high-quality multimodal monitoring data into the pre-trained causal reasoning model, identify anomalies in different dimensions, and generate path offset status data. S34. Extract abnormal features from part assembly status data, torque status data, and path offset status data; analyze the causal relationship between abnormalities based on a pre-trained causal reasoning model; and generate abnormal causal association data. S35. Based on the abnormal causal relationship data, set evaluation indicators, use the analytic hierarchy process to calculate the comprehensive impact weight of each abnormality, and generate comprehensive abnormality evaluation data.

6. The AI-based error-proofing monitoring method for flexible production lines of new energy vehicles based on multimodal perception according to claim 1, characterized in that, The S4 includes: S41. Based on production line monitoring timestamp data, filter key time nodes during the period when anomalies occur, set the time interval and scanning range of 3D laser scanning, and generate 3D laser scanning timing control data. S42. Based on the three-dimensional laser scanning timing control data, start the laser radar scanning equipment to dynamically scan the flexible production line of new energy vehicles, obtain the spatial point cloud information of the production line at each time node, and generate three-dimensional point cloud data of the production line by fusing multiple frames of point cloud data through the point cloud stitching algorithm. S43. Spatial matching of the anomaly comprehensive assessment data with the production line 3D point cloud data, marking the point cloud area corresponding to the anomaly, and combining the production line CAD model to determine the specific location and impact range of the anomaly, and generating anomaly condition activity data. S44. Extract key features from abnormal operating condition activity data, combine them with production line equipment parameters and process requirements, organize them into input feature vectors for the risk prediction model, and generate risk prediction input data. S45. Input the risk prediction input data into a pre-trained deep learning risk prediction model to predict the subsequent problems that anomalies may cause and generate mistake-proof risk prediction data.

7. The AI-based error-proofing monitoring method for flexible production lines of new energy vehicles based on multimodal perception as described in claim 6, characterized in that, S42 includes: S421. Based on the three-dimensional laser scanning timing control data, extract key control parameters and generate start-up and readiness data for the lidar equipment. S422. Based on the LiDAR device startup readiness data, trigger the device to start dynamic scanning according to the set time sequence to generate a multi-time node original point cloud dataset. S423. Based on the original point cloud dataset at multiple time nodes, perform preprocessing operations on each frame of point cloud to generate clean single-frame point cloud data. S424. Based on clean single-frame point cloud data, the ICP stitching algorithm is called: using the point cloud data at the alarm time as the reference frame, the point cloud frames of other time nodes are matched with the reference frame for feature points, and the inter-frame distance error is minimized through iterative calculation; the point cloud data of all time nodes are gradually merged to form a continuous spatial point cloud model covering the entire abnormal period, and the preliminary stitched point cloud data is generated. S425. Based on the initial stitched point cloud data, verify the quality through two dimensions; if there are quality issues, return to S424 to readjust the number of ICP algorithm iterations; after passing the verification, finally generate the production line 3D point cloud data.

8. The AI-based error-proofing monitoring method for flexible production lines of new energy vehicles based on multimodal perception according to claim 7, characterized in that, S424 includes: Based on clean single-frame point cloud data, the alarm time point cloud frame during the period of anomaly occurrence is selected as the stitching reference frame; fine feature enhancement is performed again on the reference frame to generate enhanced point cloud data of the reference frame. Based on the enhanced point cloud data of the baseline frame, and using its feature type as a reference, the same feature extraction operation is performed on the remaining cleaned single-frame point clouds to generate a multi-frame point cloud feature point set. The initial matching module of the ICP algorithm is called to compare the feature points of each frame in the multi-frame point cloud feature point set with the feature points of the base frame augmented point cloud data one by one; the matching pairs are filtered by calculating the feature descriptor similarity, the mismatch points are eliminated, and the initial matching results of inter-frame feature points are generated. Based on the initial inter-frame feature point matching results, calculate the point cloud distance error between the current frame and the reference frame; if the error is > 0.5 mm, start the ICP algorithm iterative optimization: adjust the spatial pose of the non-reference frame, recalculate the distance between matching point pairs until the error is ≤ 0.5 mm and the algorithm converges stably, generating inter-frame optimized matching data; Based on the inter-frame optimized matching data of all frames, the non-reference frame point clouds are gradually fused into the reference frame coordinate system according to the time series: first fuse the frame before the alarm, then fuse the frame after the alarm; after the fusion is completed, a spatial point cloud model covering the entire time period of the anomaly is formed, and preliminary stitched point cloud data is generated.

9. The AI-based error-proofing monitoring method for flexible production lines of new energy vehicles based on multimodal perception according to claim 1, characterized in that, The S5 includes: S51. Extract the error prevention risk handling records of the past 5 years from the production line database, count the historical losses of different abnormal types, establish a historical risk loss weight table, and generate historical risk reference data. S52. Combining error prevention risk prediction data and historical risk reference data, set a weighted formula: Error prevention risk index = (probability of risk occurrence × 0.6) + (weight of historical similar risk losses × 0.4), substitute the data to calculate the specific value, and generate the error prevention risk index; S53. Set risk level standards based on the error prevention risk index, and set corresponding response thresholds for each level to generate risk level classification standards; S54. Compare the error prevention risk index with the risk level classification standard to generate error prevention intervention trigger instructions; S55. Based on the error prevention intervention trigger command, record relevant information and generate AI error prevention early warning data for the flexible production line of new energy vehicles.

10. An AI-based error-proofing monitoring system for flexible production lines of new energy vehicles based on multimodal perception, including: One or more processors; Memory, used to store one or more programs; Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 9.

Images