Multi-sensor data fusion control method for automatic assembly line

Abstract

The invention discloses a multi-sensor data fusion control method for an automatic assembly line, and relates to the technical field of data fusion, and the method comprises the following steps: when multi-source data fusion is carried out, firstly, data information is obtained in real time through a multi-source sensor, the obtained data information is preprocessed, and the accuracy and integrity of data are ensured; by analyzing data quality and fusion performance indexes and system robustness and environmental adaptability indexes in real time, the system can accurately detect the influence of electromagnetic interference and environmental fluctuation on data and dynamically evaluate the state. And an early warning and adjustment strategy is automatically triggered in case of abnormality, so that data fusion continuity is guaranteed, and downtime and human intervention are reduced. And when the abnormity exceeds the self-repairing capability of the system, the system can give an early warning, so that the efficient and safe operation of the production line is ensured, and the capability of dealing with sudden problems is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data fusion, and particularly relates to a multi-sensor data fusion control method for an automated assembly line. Background Art

[0002] Multi-sensor data fusion in an automated assembly line refers to integrating, analyzing, and processing data from different types of sensors to obtain more comprehensive and accurate information than a single sensor. In an automated assembly line, this technology realizes real-time monitoring and intelligent judgment of the workpiece state and operation process by fusing data from various sensors such as vision sensors, force sensors, temperature sensors, and ultrasonic sensors. The fused data can eliminate the errors and uncertainties of a single sensor, improve the system's perception ability and environmental adaptability, and enable the equipment to achieve precise assembly and control in a complex and changing environment. In practical applications, data fusion relies on algorithms such as Kalman filtering, Bayesian estimation, or deep learning models to optimize and analyze multi-source data. For example, a vision sensor provides information on the position and shape of the workpiece, while a force sensor monitors the contact force during the assembly process, thus realizing precise alignment and pressure control. This multi-dimensional information fusion improves the system's decision-making ability, helps detect faults in advance, optimize the assembly process, and improve production efficiency. At the same time, multi-sensor fusion enhances the system's robustness, and even if a certain sensor fails, the system can still operate stably through the data of other sensors.

[0003] Industrial robots are widely used in the automated assembly lines of the prior art, and their main functions are to improve production efficiency, ensure assembly accuracy, and reduce labor costs and production risks. Industrial robots undertake various tasks in the assembly line, such as welding, handling, spraying, tightening, grasping, and palletizing. They achieve precise perception and efficient control of products and the environment through fusion with multi-sensors such as vision and force sensing. In addition, industrial robots can also adapt to work scenarios with high repetition and high danger, reduce human errors, and ensure the stability and consistency of the production process, and are particularly suitable for large-scale and high-precision manufacturing industries such as the automotive, electronics, and household appliance industries.

[0004] The prior art has the following deficiencies: The automated assembly lines in automobile welding workshops are usually equipped with large processing equipment such as welding machines and induction heating equipment. These equipment will generate strong electromagnetic radiation, which will interfere with the fusion of multi-sensor data and cause a series of serious consequences, seriously threatening the safety, accuracy and production efficiency of the system. Electromagnetic interference may cause sensors to send back erroneous data, especially delayed or inaccurate data from force sensors and visual sensors, resulting in misjudgment of fusion algorithms, causing robots to apply force in the wrong position or welding misalignment. This may cause collisions during grasping and handling, damaging precision parts; welding misalignment will also weaken the structural strength of the vehicle body, affecting product quality and safety, and may even require rework or scrapping, resulting in high costs and delays.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a multi-sensor data fusion control method for an automated assembly line. By real-time analysis of data quality and fusion performance indicators and system robustness and environmental adaptability indicators, the system can accurately detect the impact of electromagnetic interference and environmental fluctuations on data, and dynamically evaluate the status. Automatically trigger warnings and adjustment strategies in the event of anomalies to ensure data fusion continuity, reduce downtime and human intervention. When the anomaly exceeds the system's self-repair capability, the system will issue a warning to ensure efficient and safe operation of the production line and improve the ability to respond to emergencies to solve the problems in the above-mentioned background technology.

[0007] In order to achieve the above object, the present invention provides the following technical solution: a multi-sensor data fusion control method for an automated assembly line, comprising the following steps: When multi-source data fusion is performed, data information is first acquired in real time through multi-source sensors, and the acquired data information is pre-processed to ensure the accuracy and completeness of the data; Extract and analyze key features of multi-source sensor data information, and use the trained machine learning model to conduct a comprehensive evaluation of the collected real-time data to determine whether it will have an adverse impact on data fusion; When adverse effects are detected in the data, an early warning is triggered immediately. At the same time, the system automatically executes dynamic adjustment strategies according to the abnormal type to ensure the continuity and accuracy of data fusion under abnormal circumstances; If no anomaly is detected, a data fusion operation will be performed to optimize and integrate multi-sensor data in real time and continuously monitor the fusion results to ensure that they are consistent with expectations.

[0008] Preferably, the data information obtained in real time by the multi-source sensors includes data quality and fusion performance indicators and system robustness and environmental adaptability indicators. The data quality and fusion performance indicators are used to evaluate the accuracy, consistency of the data in the multi-sensor system, and the reliability of the fusion process. The system robustness and environmental adaptability indicators are used to evaluate the stability and response ability of the system when affected by external environmental changes.

[0009] Preferably, key feature extraction and analysis are performed on the data information obtained in real time by the multi-source sensors, including data quality and fusion performance indicators and system robustness and environmental adaptability indicators. After analyzing and processing the data quality and fusion performance indicators, a data fusion uncertainty index is generated. After analyzing and processing the system robustness and environmental adaptability indicators, an environmental interference sensitivity index is generated. The data fusion uncertainty index and the environmental interference sensitivity index after analysis are input into a pre-trained machine learning model, and a data fusion stability index is generated through the machine learning model. Whether the obtained data will have an adverse impact on the fusion process is judged through the data fusion stability index.

[0010] Preferably, the data fusion stability index generated during the data fusion process is compared and analyzed with a pre-set reference threshold of the data fusion stability index to judge whether the data collected by the multi-source sensors will have an adverse impact on the data fusion process. The judgment results are as follows: If the data fusion stability index is greater than or equal to the pre-set reference threshold of the data fusion stability index, it is determined that the data collected by the multi-source sensors has a serious adverse impact on the data fusion process, and a serious impact on the fusion state is generated; If the data fusion stability index is less than the pre-set reference threshold of the data fusion stability index, it is determined that the data collected by the multi-source sensors has not had an adverse impact on the data fusion process, and a normal fusion state is generated.

[0011] Preferably, real-time analysis is performed on the multi-source data fusion after measures are taken. The specific process is as follows: A number of data fusion stability indexes generated subsequently by the multi-source data fusion after stability measures are taken are obtained to establish an analysis set. The data fusion stability indexes in the analysis set are compared with a first-level reference threshold, a second-level reference threshold, and a reference threshold of the data fusion stability index. Among them, the second-level reference threshold is greater than the first-level reference threshold, and the first-level reference threshold is greater than the reference threshold of the data fusion stability index. The data fusion stability index is compared and analyzed with the second-level reference threshold, the first-level reference threshold, and the reference threshold of the data fusion stability index. The number of data fusion stability indexes that are less than the first-level reference threshold and greater than or equal to the reference threshold of the data fusion stability index is marked as , the number of data fusion stability indices that are less than the second-level reference threshold and greater than or equal to the first-level reference threshold is calibrated as , the number of data fusion stability indices that are greater than or equal to the second-level reference threshold is calibrated as ; Combine , , for comprehensive analysis to generate a fusion safety factor , and the formula based on is: , where in the formula, , , are , , 's preset proportionality coefficients, and , , are all greater than 0.

[0012] Preferably, compare and analyze the generated fusion safety factor with a preset fusion safety factor reference threshold, and the analysis results are as follows: If the fusion safety factor is less than the preset fusion safety factor reference threshold, a data stability signal is generated, indicating that the multi-source data is in a stable state and data fusion can be performed; If the fusion safety factor is less than the preset fusion safety factor reference threshold, a data instability signal is generated, indicating that the multi-source data is in an unstable state and data fusion cannot be performed.

[0013] Preferably, analyze and process the data quality and fusion performance indicators, and the logic for generating a data fusion uncertainty index is as follows: Under the detection window, collect data streams through multi-source sensors, perform time synchronization on them, and label the collected sampled data as , where , where in the formula, represents the sampled data of the th sensor at time , is the total number of sensors; The collaborative deviation tensor is used to measure the coupling and mutual influence between multi-sensor data, and detect whether there is inconsistency or conflict between the data. The calculation expression of the collaborative deviation is as follows: , where in the formula, is the collaborative deviation, which is the collaborative deviation between the th and the th sensors at time , and are respectively the the expected normal values of the first and the To quantify the uncertainty of multi-source sensor data, an information entropy matrix is constructed, and the multi-source sensor information entropy is calculated. The calculation expression is as follows: , where is the information entropy, that is, the information entropy of the th sensor at time , is the probability density, that is, the probability of the sensor data appearing within the discrete interval at time . The probability density is calculated as follows: , where is the number of times the data appears in the th interval within time , is the total number of data points collected within time , is the total number of discrete intervals; To capture the dynamic changes and correlations of multi-sensor data within the detection window, a multi-dimensional covariance tensor is calculated. The change in covariance reflects the stability of the data relationship between different sensors. The calculation expression is as follows: , where is the covariance tensor, representing the data covariance tensor between the th and the th sensors within the detection window , and are the average values of the th and the th sensors within the time window respectively, is the sampled data of the th sensor at time , and are the start time and the end time of the detection window respectively; Based on the information entropy , the collaborative deviation and the covariance tensor , a data fusion uncertainty index is generated. The calculation expression is as follows: , where is the data fusion uncertainty index, , and are the information entropy, 、Cooperative deviation and covariance tensor weight parameters.

[0014] Preferably, it is characterized in that the logic for analyzing and processing the system robustness and environmental adaptability indicators to generate the environmental interference sensitivity index is as follows: Under the detection window, environmental variable data and system state data are collected in real time by multi-source sensors, where is the environmental variable data set, an environmental variable data set composed of environmental variables, is the environmental variable data of the th type collected at time ; is the system state data set, a system state data set composed of system states, is the system state data of the th type collected at time ; Calculate the change rates of environmental variables and system state parameters to evaluate the response speed of the system to environmental fluctuations. The changes of different variables and state parameters are not only simple differences, but also need to consider the influence of time factors and non-linear fluctuations. The calculation expression of the environmental change rate is as follows: , where in the formula, is the environmental change rate within , is the value of the th environmental variable at time , representing the environmental variable value at the end of the detection window, is the value of the th environmental variable at time , representing the environmental variable value at the end of the detection window, is the exponential decay factor, used to control the influence of the th environmental variable in the calculation of the change rate; The calculation expression of the system state response change rate is as follows: , where in the formula, is the system state response change rate within , is the value of the th system state parameter at time , representing the system state parameter value at the end of the detection window, ​​is at time the system state parameter value at the moment, representing the system state parameter value at the end of the detection window; To evaluate the direct impact of environmental changes on the system state, a coupling sensitivity formula is introduced, and the calculation expression is as follows: , where in the formula, is the coupling sensitivity; Combined with the coupling sensitivity , the system state response change rate and the environmental change rate to generate the environmental interference sensitivity index, and the calculation expression is as follows: , where in the formula, is the environmental interference sensitivity index, is the balance coefficient, used to adjust the impact of the difference between environmental changes and system responses on the index, is a small constant to prevent the denominator from being zero, ensuring calculation stability.

[0015] In the above technical solution, the technical effects and advantages provided by the present invention are as follows: Through the real-time analysis and evaluation of data quality and fusion performance indicators, as well as system robustness and environmental adaptability indicators, the system can accurately detect and judge whether multi-source sensor data is affected by electromagnetic interference or environmental fluctuations. With the support of the data fusion uncertainty index and the environmental interference sensitivity index, combined with the data fusion stability index generated by the machine learning model, the system can dynamically evaluate the state of the data. When an anomaly is detected, the system immediately triggers an alarm and executes an automated adjustment strategy to ensure that the data fusion process can still maintain continuity and accuracy in a complex environment. This mechanism greatly improves the stability and robustness of the system and reduces the negative impact of factors such as electromagnetic interference on the production process. Before the data anomaly exceeds the system's self-repair ability, the system will preferentially choose automated processing to reduce the number of times of human intervention by re-collecting data and optimizing the fusion algorithm. This design not only reduces the downtime in production but also ensures the continuity and efficiency of production. When the fusion safety factor is lower than the preset threshold, the system will trigger an alarm to notify the operator to intervene in time to prevent the problem from further expanding. Through the above adaptive adjustment and warning mechanism, the system effectively reduces the complexity of manual operations and improves the production line's ability to respond to sudden problems, ensuring the efficient and safe operation of the automotive welding workshop. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0017] Figure 1 This is the method flowchart of a multi-sensor data fusion control method for an automated assembly line according to the present invention. Detailed implementation manners

[0018] Now, example embodiments will be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that this disclosure will be more thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art.

[0019] The present invention provides a multi-sensor data fusion control method for an automated assembly line as shown in Figure 1 the following, which includes the following steps: When performing multi-source data fusion, first, data information is obtained in real time through multi-source sensors, and the obtained data information is preprocessed to ensure the accuracy and integrity of the data; During the multi-source data fusion process, the multi-source sensors and multi-source data respectively cover various types, and are used to obtain comprehensive and accurate information in application scenarios such as automated assembly lines and industrial robots. The following are the specific compositions and detailed descriptions of the multi-source sensors and multi-source data: I. Types and functions of multi-source sensors Multi-source sensors refer to sensors that obtain data through different technologies and cover information in multiple dimensions: 1. Vision sensors (Cameras / Vision Sensors): Types: CCD / CMOS cameras, depth cameras, 3D scanners, etc. Functions: Obtain the shape, color, position, and movement state of workpieces for target recognition, positioning, and defect detection.

[0020] 2. Force / torque sensors (Force / Torque Sensors): Functions: Detect the magnitude and direction of the force exerted by the robot when performing tasks to ensure the stability and safety of operations, such as tightening screws and grasping actions 3. Temperature sensors (Temperature Sensors): Functions: Monitor the temperature of equipment or workpieces to ensure that the welding or processing process meets the process requirements and avoid overheating or insufficient cooling.

[0021] 4. Ultrasonic Sensors: Function: Used to measure distance and detect obstacles to ensure the accuracy of robot path planning and obstacle avoidance.

[0022] 5. Laser Rangefinders: Function: Provide accurate distance data for positioning and contour scanning, and are commonly used in precision measurements in industrial automation.

[0023] 6. Inertial Measurement Unit (IMU): Function: Measure angular velocity and acceleration to monitor the motion state and stability of the robot.

[0024] 7. Environmental Sensors: Types: Humidity sensors, barometric pressure sensors, etc. Function: Monitor the workshop environmental conditions to ensure process stability and product quality.

[0025] II. Classification and Content of Multi-source Data The multi-source data collected by the above sensors cover multiple dimensions and provide a basis for data fusion: 1. Visual Data: Content: Images, video streams, depth information, contour features Applications: Used for object recognition, defect detection, and path planning.

[0026] 2. Force Sensing Data: Content: Feedback data on the magnitude, direction, and contact point of the applied force Applications: Ensure that the robot applies appropriate force during assembly and welding processes to prevent damage to workpieces.

[0027] 3. Temperature Data: Content: Current temperature, temperature rise change trend Applications: Used to monitor the temperature of equipment and workpieces to avoid process problems caused by abnormal temperature.

[0028] 4. Distance and Position Data: Content: Distance, displacement, angle, direction information Applications: Support path planning, precise positioning, and obstacle avoidance operations.

[0029] 5. Motion Data: Content: Speed, acceleration, angular velocity of the robot or equipment Application: Monitor the smoothness and status of the robot's movement to prevent instability or deviation from the path.

[0030] 6. Environmental data: Content: Temperature and humidity, air pressure, environmental noise Application: Ensure that the workshop environmental conditions meet the production requirements and avoid the impact of environmental factors on product quality.

[0031] The synergistic effect of multi-source sensors and multi-source data enables the data fusion system to accurately perceive and judge in a complex environment. The multi-dimensional data provided by various sensors such as vision, force sensing, temperature, and ultrasonic waves can eliminate noise and reduce redundancy after preprocessing, ensuring the integrity and accuracy of the data. Through the multi-source data obtained in real time by these multi-source sensors, the system can monitor the production process more precisely and achieve efficient and safe automated operation.

[0032] The data information obtained in real time through multi-source sensors includes data quality and fusion performance indicators, and system robustness and environmental adaptability indicators. The data quality and fusion performance indicators are used to evaluate the accuracy, consistency of the data in the multi-sensor system, and the reliability of the fusion process. The system robustness and environmental adaptability indicators are used to evaluate the stability and response ability of the system when affected by external environmental changes (such as electromagnetic interference, temperature and humidity fluctuations, noise, etc.); Extract and analyze the key features of the multi-source sensor data information, analyze the preprocessed multi-source sensor data information, and use the trained machine learning model to comprehensively evaluate the real-time data collected to determine whether it will have an adverse impact on data fusion; Extract and analyze the key features of the data information obtained in real time through multi-source sensors, including data quality and fusion performance indicators, and system robustness and environmental adaptability indicators. After analyzing and processing the data quality and fusion performance indicators, generate a data fusion uncertainty index. After analyzing and processing the system robustness and environmental adaptability indicators, generate an environmental interference sensitivity index. Input the analyzed data fusion uncertainty index and environmental interference sensitivity index into the pre-learned machine learning model, and generate a data fusion stability index through the machine learning model. Determine whether the obtained data will have an adverse impact on the fusion process through the data fusion stability index; During the multi-source data fusion process, the changes in data quality and fusion performance metrics may be affected by electromagnetic interference. Electromagnetic interference can cause errors, delays, or packet loss in the data acquisition and transmission of sensors, especially having a particularly obvious impact on force sensors and vision sensors. The high-precision data provided by these sensors is crucial for the precise control of robots. However, under interference conditions, the vision sensor may experience image distortion, incorrect feature point recognition, or a decrease in frame rate, while the signal of the force sensor may become unstable, resulting in inaccurate force application data or response delays. This makes it difficult for the fusion algorithm to accurately determine the position and force application requirements between the robot and the workpiece when processing inconsistent or lagging data, leading to the robot applying force at the wrong position or welding misalignment. For example, during workpiece handling, data fusion errors may cause the robot to fail to grasp the target component or have a deviation in the grasping angle, resulting in the workpiece falling or being damaged; in the welding task, solder joint misalignment will weaken the body structure strength, affect product quality and safety, and may cause rework or scrapping. Therefore, the impact of electromagnetic interference on multi-source data fusion not only reduces the judgment accuracy and control stability of the system but also may directly threaten the safety and efficiency of the production line, causing serious economic losses.

[0033] The logic for analyzing and processing data quality and fusion performance metrics to generate a data fusion uncertainty index is as follows: Under the detection window, data streams are collected by multi-source sensors, time-synchronized, and the collected sampled data is calibrated , where , in the formula, represents the sampled data of the th sensor at time , is the total number of sensors; The collaborative deviation tensor is used to measure the coupling and mutual influence between multi-sensor data and detect whether there is inconsistency or conflict between the data. The calculation expression of the collaborative deviation is as follows: , in the formula, is the collaborative deviation, which is the collaborative deviation between the th and the th sensors at time , and are the expected normal values of the th and the th sensors respectively; The expected normal values of sensors are usually determined in various ways to ensure that they can reflect the ideal output of the system under normal operating conditions. First, statistical analysis can be performed based on historical data, recording the typical values and stable ranges of sensors during long-term operation to establish a reference model. Second, system calibration experiments can be carried out, that is, measuring the sensor output under known environmental and standard conditions to obtain accurate calibration values. For dynamic scenarios, physical simulation models or data-driven machine learning models can be used to predict the normal value range. In addition, when the environment is complex and variable, adaptive algorithms can be introduced to update the expected values in real time to ensure that they change with the operating conditions. Through these methods, the expected normal values provide a reliable benchmark for sensor data fusion to identify abnormal situations and interference effects.

[0034] To quantify the uncertainty of multi-source sensor data, an information entropy matrix is constructed, and the multi-source sensor information entropy is calculated. The calculation expression is as follows: , where is the information entropy, that is, the information entropy of the th sensor at time , is the probability density, that is, the probability that the sensor data appears in the time interval, reflecting the distribution characteristics of the output data of this sensor. Among them, the probability density is calculated as follows: , where is the number of times the data appears in the th interval within time , is the total number of data points collected within time , is the total number of discrete intervals; Uncertainty and its impact of multi-source sensors In a multi-source sensor system, the output of each sensor may be affected by various internal and external factors, resulting in data uncertainty. This uncertainty not only affects the accuracy of the sensor itself but also the results of multi-sensor data fusion, thereby weakening the overall performance and stability of the system. The following elaborates in detail on the sources of uncertainty of multi-source sensors and their possible impacts.

[0035] 1. Uncertainty of internal noise of sensors Description: The electronic components of each sensor will generate certain noise, such as thermal noise, random drift, or quantization noise. These noises will cause slight fluctuations in the output value of the sensor under the same input conditions.

[0036] Impact: If the noise is not effectively filtered, it will cause subtle error accumulation during multi-sensor data fusion, which will lead to inaccurate fusion results. This is particularly fatal in high-precision tasks such as welding path control or high-precision grasping.

[0037] 2. Uncertainties Caused by Environmental Interference Description: Factors such as electromagnetic interference, temperature and humidity fluctuations, and vibrations in the environment will affect the output of sensors. For example, strong electromagnetic fields will interfere with the data transmission of force sensors or wireless communication modules.

[0038] Influence: If the multi-source sensor system is sensitive to these external interferences, it may lead to inconsistent or ineffective data fusion, ultimately resulting in incorrect decisions by the robot, such as failed grasping during handling or misaligned welding.

[0039] 3. Calibration Error and Drift Uncertainty Description: During long-term use, the performance of sensors will gradually degrade, resulting in calibration failure, or the sensors will drift, that is, the data will gradually deviate from the true value without control.

[0040] Influence: Calibration errors will cause the output values of sensors to deviate from the normal range for a long time, thus leading to misjudgments in the fusion algorithm. Especially in dynamic fusion, if the calibration values are not updated in time, the robot may perform incorrect operations without knowing it.

[0041] 4. Delay and Time Synchronization Uncertainty Description: There are different delays in the data transmission of each sensor. Especially in wireless sensor networks, the delay differences will cause the data to be out of sync on the time axis.

[0042] Influence: Poor time synchronization will cause the fusion algorithm to calculate based on outdated data, resulting in a lag in the response of the control system to environmental changes, which may cause position deviations or motion errors in robot operations.

[0043] 5. Data Loss and Transmission Uncertainty Description: In a complex workshop environment, sensor data may be lost or delayed during transmission due to network congestion, electromagnetic interference, etc.

[0044] Influence: Data loss will cause the fusion algorithm to lack complete information at critical moments, resulting in misjudgments. The system may still execute tasks with insufficient data, increasing the risk of operation failure.

[0045] 6. Uncertainty of Sensor Redundancy Conflict Description: When multiple sensors monitor the same target, conflicts or inconsistencies may occur among the data. For example, visual sensors and ultrasonic sensors may give different distance data when detecting the position of an object.

[0046] Impact: This kind of inconsistency increases the complexity of the fusion algorithm. The system must decide which data is more reliable through weighted or screening algorithms, increasing the processing time and even potentially causing decision-making errors.

[0047] 7. Uncertainty of Dynamic Scenes Description: In a dynamic environment, such as when a robot collaborates with humans or other devices, sensors may not be able to capture the rapid changes in the environment in a timely manner, resulting in incomplete or delayed data.

[0048] Impact: The uncertainty in dynamic scenes causes the robot to be insensitive to real-time changes, such as failing to avoid employees in a timely manner or misjudging the position of an object, increasing operational risks and safety hazards.

[0049] 8. Uncertainty of System Model Error Description: Multi-sensor systems rely on mathematical models or machine learning models to predict and process sensor data, but the models themselves may have errors, especially in situations that have not been encountered or in extreme environments.

[0050] Impact: Model errors will amplify the problems in sensor data, leading the system to make unreasonable operations. For example, a robot may misidentify the target object or apply the wrong force due to misjudgment.

[0051] The sources of uncertainty in multi-source sensors are extensive, including internal noise, environmental interference, calibration errors, poor time synchronization, data loss, redundancy conflicts, dynamic scene changes, and system model errors. These uncertainties will have varying degrees of impact on the results of sensor data fusion, such as misjudgment, delay, inconsistency, or decision-making errors. In an automated assembly line, failure to handle these uncertainties in a timely manner may lead to robot operation errors, product quality degradation, or even production line shutdown. Therefore, the system needs to adopt means such as adaptive algorithms, redundant design, and real-time monitoring to cope with uncertainties and ensure the accuracy of sensor data fusion and the stability of system operation.

[0052] To capture the dynamic changes and correlations of multi-sensor data within the detection window, a multi-dimensional covariance tensor is calculated. The change in covariance reflects the stability of the data relationship between different sensors. The calculation expression is as follows: , where is the covariance tensor, representing the data covariance tensor between the th and the th sensors within the detection window . The value of the covariance tensor reflects whether the outputs of the two sensors change synchronously and their degree of coupling, and are the average values of the th and the th sensors within the time window respectively, is the sampled data of the th sensor at time , and are the start time and the end time of the detection window respectively; In multi-sensor data fusion, the system captures the dynamic changes and correlations of sensor data within a specific time period through the detection window. These changes and correlations reflect the real-time response of the sensors to the environment and system operations, and help the fusion algorithm analyze the consistency and reliability of the sensor data. The following details the possible dynamic changes and correlations of multi-sensor data within the detection window: 1. Data trend changes Description: Sensors may show some trend changes within the detection window, such as a gradual increase in temperature, a gradual increase in force sensing value, or a gradual decrease in distance measurement value. Trend changes indicate that the system is performing a specific operation, such as the temperature rising during the welding process, or the pressure gradually increasing during the handling process.

[0053] Correlation: Positive correlation: The temperature sensor changes synchronously with the welding current.

[0054] Negative correlation: The visual distance sensor is negatively correlated with the speed of the robot approaching the workpiece (as the distance decreases, the speed increases).

[0055] Impact: Failure to capture these trends in a timely manner may lead to misjudgment of the system state and the inability to respond to abnormal situations in advance, such as overheating during welding or insufficient force.

[0056] 2. Periodic changes in data Description: Some sensor data may exhibit periodic fluctuations within the detection window, such as speed sensors of rotating motors, frequency sensors, or vibration sensors in robot motion. The periodic changes reflect the periodic operating characteristics of the device or system.

[0057] Correlation: Periodic coordination between speed and position sensors (the speed change period is consistent with the position change period).

[0058] The vision sensor and the force sensing sensor cooperate to adjust, reflecting the robot's periodic grasping and placing tasks.

[0059] Impact: If abnormal periodic data or a disordered period occurs, it may indicate a malfunction in the system operation, such as motor out-of-step or unstable robot motion.

[0060] 3. Data Noise and Instantaneous Fluctuations Description: Data noise and instantaneous fluctuations are random fluctuations in the sensor output, which may be caused by electromagnetic interference, temperature fluctuations, or system mechanical vibrations. The noise data will mask the actual changes and interfere with the system's judgment.

[0061] Correlation: Noise may occur simultaneously in different sensors, such as electromagnetic interference causing fluctuations in the data of multiple sensors.

[0062] When the noise is inconsistent, the system can reduce the impact through multi-source data fusion (such as filtering algorithms).

[0063] Impact: If the system fails to detect and process the noise in a timely manner, it may lead to failure of robot operations, such as abnormal force in grasping actions or deviation of welding paths.

[0064] 4. Data Delay and Time Synchronization Error Description: Data acquisition and transmission of multiple sensors may have time delays or be out of sync. This is especially common in wireless sensor networks and can cause data fusion algorithms to misjudge the current state.

[0065] Correlation: Data delay will reduce the correlation between sensors, such as the data of the force sensor lagging behind that of the vision sensor.

[0066] By time synchronization (such as the PTP protocol), the impact of delay on data fusion can be reduced.

[0067] Impact: Time synchronization error may cause the robot to apply force or weld at the wrong time point, resulting in operation errors.

[0068] 5. Abnormal Data and Data Conflicts Description: Some sensors may output abnormal data within the detection window, or data conflicts may occur among multiple sensors, such as inconsistent object positions given by vision sensors and distance sensors.

[0069] Relevance: Abnormal data or conflicting data can make it difficult for the fusion algorithm to determine which sensor data is more reliable.

[0070] The system can handle these conflicts through data screening and weighting strategies.

[0071] Impact: Failure to handle data conflicts in a timely manner can lead to misoperations of the robot, such as welding at the wrong position or failed grasping.

[0072] 6. Data Drift Description: Data drift is a phenomenon where the values output by sensors gradually deviate from the true values over a long period of time, usually caused by sensor aging, environmental changes, or inaccurate calibration.

[0073] Relevance: Data drift can cause small but cumulative deviations between the outputs of multiple sensors, such as the positioning data of force sensing sensors and vision sensors no longer matching.

[0074] The system can reduce the impact of drift through periodic calibration or adaptive calibration algorithms.

[0075] Impact: Data drift may accumulate unnoticed, leading to a decline in product quality or unstable operation.

[0076] 7. Nonlinear Variation of Data Description: The outputs of some sensors will exhibit a nonlinear relationship under specific conditions, such as the output of a temperature sensor becoming inaccurate at extreme temperatures, or the image quality of a vision sensor deteriorating in low light.

[0077] Relevance: Nonlinear variation makes it more difficult for the fusion algorithm to predict system behavior, especially in complex operations.

[0078] The system can use machine learning models to process nonlinear data.

[0079] Impact: Failure to correctly process nonlinear data may lead to incorrect robot operation paths or inaccurate force application.

[0080] 8. Emergency and Anomaly Detection Description: Within the detection window, the system may capture emergencies such as sensor failures, communication interruptions, or unexpected environmental changes. These events can cause data to suddenly deviate from the normal range.

[0081] Relevance: The system should detect emergencies in real time and promptly enable redundant data sources.

[0082] Multi-source data fusion helps identify single sensor failures and avoid incorrect operations.

[0083] Impact: Failure to respond to emergencies in a timely manner may lead to system downtime, product scrapping, or safety accidents.

[0084] The dynamic changes of multi-sensor data within the detection window include trend changes, periodic fluctuations, noise interference, delays, data conflicts, drifts, non-linear changes, and emergencies, etc. These changes reflect the responses of sensors to the operating environment and task status. By detecting these dynamic changes and their relevance, the system can optimize the data fusion strategy, improve the accuracy and robustness of decision-making, and ensure the safety and efficiency of automated production.

[0085] Based on information entropy , collaborative deviation and covariance tensor Generate a data fusion uncertainty index, and the calculation expression is as follows: , where is the data fusion uncertainty index, , and are the weight parameters of information entropy , collaborative deviation and covariance tensor respectively; It can be seen from the calculation expression of the data fusion uncertainty index that under the detection window, the larger the value of the data fusion uncertainty index generated after analyzing and processing the data quality and fusion performance indicators, the greater the inconsistency, delay, or error between the data collected by the sensors, and it is difficult for the fusion algorithm to accurately judge the reliability of the information, which implies that the data may be affected by electromagnetic interference and the probability of electromagnetic interference is higher. On the contrary, when the value of the data fusion uncertainty index is smaller, it means that the data consistency between sensors is good, the noise is low, the response is timely, the fusion performance is high, the system's trust in the data is improved, and the possibility of electromagnetic interference is small. Therefore, the level of the data fusion uncertainty index value directly reflects the probability and degree of the impact of electromagnetic interference.

[0086] During the multi-source data fusion process, if the data quality and fusion performance metrics change, it may indicate that the data information collected by multi-source sensors has been affected by electromagnetic interference. Electromagnetic interference can cause delays, packet losses, distortions, or increased noise during the sensor data transmission process, especially having a significant impact on force-sensing and vision sensors. Force-sensing sensors may provide incorrect contact force or force application information due to interference, causing the robot to apply excessive or insufficient force when performing grasping or welding tasks. When vision sensors are affected by electromagnetic interference, image data distortion, frame loss, or recognition delays may occur, affecting the accurate judgment of the workpiece's position and shape. Since multi-sensor data fusion relies on the synchronization and accuracy of each sensor's information, any problem with a key data source will cause the fusion algorithm to make incorrect judgments. For example, if the vision sensor provides incorrect position information due to interference and combines it with the delayed data from the force-sensing sensor, it may cause the system to incorrectly calculate the position and force that the robot should apply, resulting in applying force at the incorrect position or misaligned welding. This not only affects product quality but may also cause damage to precision components or production line stoppages, seriously threatening the system's safety, accuracy, and production efficiency.

[0087] The logic for analyzing and processing the system robustness and environmental adaptability metrics to generate the environmental interference sensitivity index is as follows: Under the detection window, real-time environmental variable data is collected through multi-source sensors and system status data , where, , is the environmental variable data set, an environmental variable data set composed of environmental variables, is the th environmental variable data collected at time , , is the system status data set, a system status data set composed of system statuses, is the th system status data collected at time ; Detailed classification and functions of environmental variables and system status data I. Environmental variable data Environmental variable data describes the external environmental conditions in which the system is located. These factors can directly or indirectly affect the sensor performance of the system, the accuracy of data fusion, and the operating stability of automated equipment. The following are typical environmental variables and their detailed explanations: 1. Electromagnetic interference intensity Description: Welding machines, induction heating equipment, etc. in the workshop will generate strong electromagnetic radiation, interfering with the signals of electronic devices and sensors.

[0088] Impact: It will cause the loss and distortion of sensor data, especially having a significant impact on vision sensors and wireless communication systems.

[0089] 2. Temperature Description: The temperature in the workshop may fluctuate due to machine operation or environmental climate.

[0090] Impact: A high-temperature environment may cause the performance degradation of some sensors (such as laser sensors and force-sensing sensors), increasing the error of data.

[0091] 3. Humidity Description: A high-humidity environment may affect the reliability and lifespan of electronic components.

[0092] Impact: Humidity will interfere with optical devices, such as causing fogging of the lens or reducing the performance of laser sensors.

[0093] 4. Noise interference Description: Sound waves or ultrasonic waves in the workshop may interfere with sensor signals.

[0094] Impact: Ultrasonic sensors are easily affected by environmental noise, resulting in ranging errors and interfering with system judgment.

[0095] 5. Vibration Description: The mechanical vibration generated during the operation of large equipment will affect the reading stability of sensors.

[0096] Impact: It will interfere with high-precision force sensors and laser ranging sensors, resulting in data jitter or inaccuracy.

[0097] 6. Light intensity Description: The lighting conditions in the workshop may vary with time and area.

[0098] Impact: Insufficient or excessive light will affect the recognition ability of vision sensors, resulting in misjudgment of the vision system.

[0099] 7. Air quality (dust concentration) Description: The smoke during the welding process or the dust in the workshop will affect the performance of optical devices.

[0100] Impact: The detection accuracy of laser sensors or vision systems will decrease, and may even cause equipment failures.

[0101] II. System status data System status data reflects the internal operating conditions of the system, describing the health, response capabilities, and data transmission status of various sensors and control systems. The following are common system status data and their detailed explanations: 1. Sensor response time Description: The time required for a sensor to detect a signal and generate data.

[0102] Impact: A longer response time may cause data delay and affect the real-time control ability of the system.

[0103] 2. Signal integrity Description: Whether data is distorted, packet lost, or interfered by noise during transmission.

[0104] Impact: Poor signal integrity will lead to inaccurate data fusion and thus affect the control accuracy of the robot.

[0105] 3. Data transmission rate Description: The data transmission speed between the sensor and the control system.

[0106] Impact: A lower transmission rate will result in insufficient real-time performance and delay the system's response.

[0107] 4. Packet loss rate Description: The number of lost data packets during the data transmission between the sensor and the system.

[0108] Impact: A high packet loss rate will lead to incomplete data and increase the error of data fusion.

[0109] 5. Sensor calibration status Description: Whether the sensor is in the correct calibration state.

[0110] Impact: Uncalibrated or malfunctioning sensors will generate incorrect data and affect system judgment.

[0111] 6. Noise level Description: The noise intensity of the internal signals of the system.

[0112] Impact: A high noise level will increase the uncertainty of data and reduce the performance of the fusion algorithm.

[0113] 7. Power consumption level Description: The energy consumption of each module of the system.

[0114] Impact: Abnormal power consumption may indicate that some devices are overloaded or malfunctioning.

[0115] 8. Load rate of the control system Description: The current load situation of the control system.

[0116] Impact: High load may cause the system to respond sluggishly, reducing data processing and fusion efficiency.

[0117] 9. Redundant Sensor Status Description: Whether the backup sensor is working properly.

[0118] Impact: Abnormal status of redundant sensors may lead to a lack of effective alternatives in case of sensor failures.

[0119] Environmental variable data: The status of the external environment, such as electromagnetic interference, temperature and humidity, light intensity, etc., directly affects the performance of sensors and the stability of the system.

[0120] System status data: The internal operating conditions of the system, such as sensor response time, data transmission rate, signal integrity, etc., determine the system's response ability to environmental changes and the accuracy of data fusion.

[0121] By monitoring environmental variables and system status data in real time, the system can dynamically adjust the data fusion strategy in a harsh environment, improve robustness and adaptability, and ensure the stable and efficient operation of the production process.

[0122] Calculate the change rates of environmental variables and system status parameters to evaluate the system's response speed to environmental fluctuations. The changes of different variables and status parameters are not only simple differences, but also need to consider the influence of time factors and non-linear fluctuations. The calculation expression of the environmental change rate is as follows: , where is the environmental change rate within is the value of the th environmental variable at time , representing the environmental variable value at the end of the detection window, is the value of the th environmental variable at time , representing the environmental variable value at the end of the detection window, is the exponential decay factor, used to control the influence of the th environmental variable in the change rate calculation; The calculation expression of the system status response change rate is as follows: , where is the system status response change rate within is the value of the th system status parameter at time , representing the system status parameter value at the end of the detection window, is the value of the th system status parameter at time The numerical value of the system state parameter, representing the numerical value of the system state parameter at the end of the detection window; To evaluate the direct impact of environmental changes on the system state, a coupling sensitivity formula is introduced, and the calculation expression is as follows: , where is the coupling sensitivity; The coupling sensitivity is an index used to measure the degree of association between environmental variables (such as electromagnetic interference, temperature, humidity, etc.) and system state parameters (such as sensor response time, data transmission rate, signal integrity, etc.). It reflects how sensitive the system state is to small changes in environmental variables, that is, when the environment changes, to what extent the state parameters of the system will fluctuate. The coupling sensitivity captures the instantaneous association relationship between them by calculating the partial derivatives between each pair of environmental variables and state parameters, thereby quantifying the adaptability and stability of the system.

[0123] In the calculation of the environmental interference sensitivity index, the coupling sensitivity is used to evaluate the response ability and vulnerability of the system to external environmental changes. A higher coupling sensitivity indicates that the system state highly depends on environmental changes, and external disturbances will significantly affect the system performance, making it more prone to control errors or downtime. On the contrary, a lower coupling sensitivity means that the system has stronger adaptability to environmental changes and can maintain stable operation in a complex and changing environment. Therefore, the introduction of the coupling sensitivity not only helps to identify potential risk points of the system but also provides data support for system design and adjustment, making it more robust.

[0124] Combined with the coupling sensitivity , the system state response change rate and the environmental change rate to generate the environmental interference sensitivity index, and the calculation expression is as follows: , where is the environmental interference sensitivity index, is the balance coefficient, used to adjust the impact of the difference between environmental changes and system responses on the index, is a small constant to prevent the denominator from being zero and ensure calculation stability; It can be seen from the calculation expression of the environmental interference sensitivity index that under the detection window, after analyzing and processing the system robustness and environmental adaptability indicators, the larger the value of the environmental interference sensitivity index performance generated, the higher the sensitivity of the system to external environmental interference, which means that the probability of the data information collected by the multi-source sensors being affected by electromagnetic interference is greater. This may cause delays, distortions, or losses in the data fed back by the sensors, affecting the accuracy of data fusion and the normal operation of the system. On the contrary, if the value of the environmental interference sensitivity index performance is smaller, it indicates that the system exhibits strong robustness and environmental adaptability within the current detection window, the data of the multi-source sensors are more stable and reliable, and the possibility of being affected by electromagnetic interference is lower.

[0125] The machine learning model is not limited here, and it can realize the data fusion uncertainty index and the environmental interference sensitivity index After comprehensive analysis, any machine learning model that can generate the data fusion stability index is acceptable. To implement the technical solution of the present invention, the present invention provides a specific implementation method; The generation logic of the data fusion stability index is as follows: , where , are the preset proportionality coefficients of the data fusion uncertainty index and the environmental interference sensitivity index , and , are both greater than 0.

[0126] It can be seen from the calculation expression of the data fusion stability index that under the detection window, the larger the value of the data fusion uncertainty index performance generated after analyzing and processing the data quality and fusion performance indicators, and the larger the value of the environmental interference sensitivity index performance generated after analyzing and processing the system robustness and environmental adaptability indicators, the larger the value of the data fusion stability index performance, indicating that the probability of the data information collected by the multi-source sensors being affected by electromagnetic interference is greater, and vice versa, indicating that the probability of the data information collected by the multi-source sensors being affected by electromagnetic interference is smaller.

[0127] Compare and analyze the data fusion stability index generated during the data fusion process with the pre-set reference threshold of the data fusion stability index to determine whether the data collected by the multi-source sensors will have an adverse impact on the data fusion process. The judgment results are as follows: If the data fusion stability index is greater than or equal to the pre-set reference threshold of the data fusion stability index, it is determined that the data collected by the multi-source sensors has a serious adverse impact on the data fusion process, and a serious impact on the fusion state is generated; If the data fusion stability index is less than the pre-set reference threshold of the data fusion stability index, it is determined that the data collected by the multi-source sensors has no adverse impact on the data fusion process, and a normal fusion state is generated; When adverse effects on the data are detected, an alarm is immediately triggered. At the same time, the system automatically executes a dynamic adjustment strategy according to the type of anomaly to ensure the continuity and accuracy of data fusion under abnormal conditions; Dynamic adjustment strategy for data fusion uncertainty When it is detected that the data fusion uncertainty index is too high, indicating that there are inconsistencies or delays between the sensor data, the system automatically executes the following adjustment strategies to reduce uncertainty and ensure the continuity and accuracy of the data fusion process. First, the system will reallocate the fusion weights of each sensor, reducing the weight of the sensor with greater interference or delay and increasing the weights of other sensors (such as redundant data sources). Second, the system will apply an adaptive Kalman filter or time synchronization algorithm to correct the errors caused by asynchronous data timestamps. In addition, if a sensor continuously feeds back abnormal data for a long time, the system will temporarily exclude the data of this sensor and enable a backup sensor for compensation. Through these dynamic adjustments, the system can maintain the stability of data fusion in a high-uncertainty environment and avoid operation errors caused by misjudgment.

[0128] Dynamic adjustment strategy for environmental interference sensitivity When the environmental interference sensitivity index increases, indicating that the system is more sensitive to environmental factors such as external electromagnetic interference and temperature and humidity changes, the system will immediately execute a series of dynamic adjustment strategies to enhance its robustness and adaptability. First, the system will automatically switch to a more robust sensor mode. For example, when the vision sensor is interfered, ultrasonic or laser rangefinder sensors will be preferentially used. Second, the system will reduce the interference of noise on the data through signal filtering algorithms (such as low-pass filters or adaptive filters). At the same time, the system will adjust the data acquisition frequency in real time to avoid high-interference periods and enhance the redundant acquisition of important sensor data. In addition, the system may automatically trigger environmental monitoring devices to analyze and alarm the interference source and notify the operator in time for necessary adjustments. These strategies ensure that the system can maintain efficient and stable operation even in a complex environment and avoid production interruptions or quality problems.

[0129] Perform real-time analysis on the multi-source data fusion after taking measures. The specific process is as follows: Obtain a number of data fusion stability indices generated subsequently from the multi-source data fusion after taking stabilization measures to establish an analysis set. Compare the data fusion stability indices in the analysis set with the first-level reference threshold, the second-level reference threshold, and the data fusion stability index reference threshold. Among them, the second-level reference threshold is greater than the first-level reference threshold, and the first-level reference threshold is greater than the data fusion stability index reference threshold. Compare and analyze the data fusion stability index with the second-level reference threshold, the first-level reference threshold, and the data fusion stability index reference threshold. Label the number of data fusion stability indices that are less than the first-level reference threshold and greater than or equal to the data fusion stability index reference threshold as , label the number of data fusion stability indices that are less than the second-level reference threshold and greater than or equal to the first-level reference threshold as , label the number of data fusion stability indices that are greater than or equal to the second-level reference threshold as ; Perform a comprehensive analysis on , , to generate a fusion safety factor , and the basis formula is: , in the formula, , , are the preset proportionality coefficients of , , , and , , are all greater than 0.

[0130] It can be seen from the calculation expression of the fusion safety factor that the larger the value of the fusion safety factor performance, the worse the effect of the system automatically executing the dynamic adjustment strategy, indicating that the probability of abnormality occurring during data fusion at this moment is greater. On the contrary, it indicates that the effect of the system automatically executing the dynamic adjustment strategy is better, indicating that the probability of abnormality occurring during data fusion at this moment is smaller.

[0131] Compare and analyze the generated fusion safety factor with the pre-set fusion safety factor reference threshold. The analysis results are as follows: If the fusion safety factor is less than the pre-set fusion safety factor reference threshold, generate a data stability signal, indicating that the multi-source data is in a stable state and data fusion can be performed; If the fusion safety factor is less than the pre-set fusion safety factor reference threshold, a data instability signal is generated, indicating that the multi-source data is in an unstable state and data fusion cannot be performed. At this time, the system preferentially selects automated processing to reduce the number of human interventions by re-collecting data to ensure production continuity. When the data anomaly exceeds the system's self-repair ability, a warning should be issued to notify the operator to intervene in a timely manner; If no anomaly is detected, the data fusion operation will be executed to perform real-time optimization and integration of multi-sensor data and continuously monitor the fusion result to ensure its compliance with the expectation; When the system does not detect anomalies in the multi-source data, the system will execute the data fusion operation, collect and optimize the data from multiple sensors in real time, and generate accurate results through the fusion algorithm. In this process, the system will use a variety of data processing algorithms, such as Kalman filtering, Bayesian inference, etc., to eliminate data noise, correct errors and improve the integrity and consistency of information. The goal of real-time optimization is to ensure that the data from each sensor complements each other, reduce the deficiencies or errors of single-sensor data, and ultimately form a more comprehensive and accurate judgment.

[0132] Meanwhile, the system will continuously monitor the fusion result, that is, after data fusion, it will also dynamically detect the output result to ensure its consistency with the preset standard or expected state. If the fusion result shows a deviation or does not meet the expectation (such as inaccurate robot movement), the system will give timely feedback and adjust the fusion strategy. For example, if the change in environmental light causes an increase in the deviation of the visual sensor data, the system will reduce the weight of the visual data and enhance the role of the force sensing data. This mechanism of continuous monitoring and dynamic adjustment ensures that robots and automated equipment can operate stably in complex and changing environments, achieve high-precision welding, handling and other tasks, and maintain the efficiency and safety of the production line.

[0133] Through the real-time analysis and evaluation of data quality and fusion performance indicators, as well as system robustness and environmental adaptability indicators, the system can accurately detect and judge whether multi-source sensor data is affected by electromagnetic interference or environmental fluctuations. With the support of the data fusion uncertainty index and the environmental interference sensitivity index, combined with the data fusion stability index generated by the machine learning model, the system can dynamically evaluate the state of the data. When an anomaly is detected, the system immediately triggers an alarm and executes an automated adjustment strategy to ensure that the data fusion process can still maintain continuity and accuracy in a complex environment. This mechanism greatly improves the stability and robustness of the system and reduces the negative impact of factors such as electromagnetic interference on the production process. Before the data anomaly exceeds the system's self-repair ability, the system will preferentially choose automated processing to reduce the number of human interventions by re-collecting data and optimizing the fusion algorithm. This design not only reduces the downtime in production but also ensures the continuity and efficiency of production. When the fusion safety factor is lower than the preset threshold, the system will trigger an alarm to notify the operator to intervene in time to prevent the problem from further expanding. Through the above adaptive adjustment and warning mechanism, the system effectively reduces the complexity of manual operations and improves the production line's ability to respond to unexpected problems, ensuring the efficient and safe operation of the automotive welding workshop.

[0134] Only some exemplary embodiments of the present invention have been described above by way of illustration. Undoubtedly, for those of ordinary skill in the art, various different ways can be used to modify the described embodiments without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A multi-sensor data fusion control method for an automated assembly line, characterized in that: The following steps are involved: When multi-source data fusion is performed, data information is first acquired in real time through multi-source sensors, and the acquired data information is pre-processed to ensure the accuracy and completeness of the data; Extract and analyze key features of multi-source sensor data information, and use the trained machine learning model to conduct a comprehensive evaluation of the collected real-time data to determine whether it will have an adverse impact on data fusion; When adverse effects are detected in the data, an early warning is triggered immediately. At the same time, the system automatically executes dynamic adjustment strategies according to the abnormal type to ensure the continuity and accuracy of data fusion under abnormal circumstances; If no anomaly is detected, a data fusion operation will be performed to optimize and integrate multi-sensor data in real time and continuously monitor the fusion results to ensure that they are consistent with expectations.

2. A multi-sensor data fusion control method for an automated assembly line according to claim 1, characterized in that: The data information obtained in real time through multi-source sensors includes data quality and fusion performance indicators and system robustness and environmental adaptability indicators. The data quality and fusion performance indicators are used to evaluate the accuracy and consistency of data in the multi-sensor system and the reliability of the fusion process. The system robustness and environmental adaptability indicators are used to evaluate the stability and response capabilities of the system when affected by changes in the external environment.

3. A multi-sensor data fusion control method for an automated assembly line according to claim 2, characterized in that: Key features of the data information acquired in real time through multi-source sensors, including data quality and fusion performance indicators and system robustness and environmental adaptability indicators, are extracted and analyzed. After analyzing and processing the data quality and fusion performance indicators, a data fusion uncertainty index is generated. After analyzing and processing the system robustness and environmental adaptability indicators, an environmental interference sensitivity index is generated. The analyzed data fusion uncertainty index and environmental interference sensitivity index are input into a pre-learned machine learning model, and a data fusion stability index is generated through the machine learning model. The data fusion stability index is used to determine whether the acquired data will have an adverse effect on the fusion process.

4. The multi-sensor data fusion control method for an automated assembly line according to claim 3, characterized in that: The data fusion stability index generated during the data fusion process is compared and analyzed with the pre-set data fusion stability index reference threshold to determine whether the data collected by the multi-source sensors will have an adverse effect on the data fusion process. The judgment results are as follows: If the data fusion stability index is greater than or equal to a preset data fusion stability index reference threshold, it is determined that the data collected by the multi-source sensor has a serious adverse impact on the data fusion process, and a serious impact fusion state is generated; If the data fusion stability index is less than a preset data fusion stability index reference threshold, it is determined that the data collected by the multi-source sensor has no adverse effect on the data fusion process, and a normal fusion state is generated.

5. A multi-sensor data fusion control method for an automated assembly line according to claim 4, characterized in that: The multi-source data fusion after measures are taken is analyzed in real time. The specific process is as follows: a number of data fusion stability indexes generated after the multi-source data fusion after stabilization measures are taken are obtained to establish an analysis set, and the data fusion stability index in the analysis set is compared with the first-level reference threshold, the second-level reference threshold and the data fusion stability index reference threshold, wherein the second-level reference threshold is greater than the first-level reference threshold, and the first-level reference threshold is greater than the data fusion stability index reference threshold, and the data fusion stability index is compared and analyzed with the second-level reference threshold, the first-level reference threshold and the data fusion stability index reference threshold, and the number of data fusion stability indexes that are less than the first-level reference threshold and greater than or equal to the data fusion stability index reference threshold is calibrated as , the number of data fusion stability indexes that are less than the second level reference threshold and greater than or equal to the first level reference threshold is calibrated as , the number of data fusion stability indexes greater than or equal to the second level reference threshold is calibrated as ; Will , , Conduct comprehensive analysis to generate integrated safety factors , based on the formula: , where , , for , , The preset scaling factor of , , Both are greater than 0.

6. A multi-sensor data fusion control method for an automated assembly line according to claim 5, characterized in that: The generated fusion safety factor is compared and analyzed with the preset fusion safety factor reference threshold. The analysis results are as follows: If the fusion safety factor is less than the preset fusion safety factor reference threshold, a data stability signal is generated, indicating that the multi-source data is in a stable state and data fusion can be performed; If the fusion safety factor is less than the preset fusion safety factor reference threshold, a data instability signal is generated, indicating that the multi-source data is in an unstable state and data fusion cannot be performed.

7. The multi-sensor data fusion control method for an automated assembly line according to claim 3, characterized in that: The logic of analyzing and processing the data quality and fusion performance indicators and generating the data fusion uncertainty index is as follows: In the detection window, data streams are collected through multi-source sensors, time-synchronized, and the collected sampled data are marked. ,in, , where Indicates Sensors at time The sampling data at the time, is the total number of sensors; The collaborative deviation tensor is used to measure the coupling and mutual influence between multi-sensor data and detect whether there is inconsistency or conflict between the data. The collaborative deviation calculation expression is as follows: , where is the collaborative deviation, and Sensors at time The coordination deviation of the moment, and They are and The expected normal value for each sensor; In order to quantify the uncertainty of multi-source sensor data, the information entropy matrix is ​​constructed and the information entropy of multi-source sensors is calculated. The calculation expression is as follows: , where is the information entropy, i.e. Sensors at time The information entropy of is the probability density, i.e., the sensor data at time Moment The probability of occurrence in a discrete interval, where the probability density The calculation is as follows: , where It's time Neidi The number of times data appears in an interval, It's time The total number of data points collected within is the total number of discrete intervals; In order to capture the dynamic changes and correlations of multi-sensor data within the detection window, the multi-dimensional covariance tensor is calculated. The change in covariance reflects the stability of the data relationship between different sensors. The calculation expression is as follows: , where is the covariance tensor, indicating the and Between sensors, within the detection window The data covariance tensor in , and They are and The average value of sensors in the time window, It is Sensors at time The sampling data at the time, and They are the detection window start time and the detection window end time; Based on information entropy , collaborative deviation and the covariance tensor Generate data fusion uncertainty index, the calculation expression is as follows: , where is the data fusion uncertainty index, , as well as Information entropy , collaborative deviation and the covariance tensor The weight parameter of .

8. The multi-sensor data fusion control method for an automated assembly line according to claim 3, characterized in that: The logic of analyzing and processing the system robustness and environmental adaptability indicators to generate the environmental interference sensitivity index is as follows: Under the detection window, real-time environmental variable data is collected through multi-source sensors and system status data ,in, , is the environment variable dataset, consisting of An environment variable dataset consisting of environment variables, It's in time The first Environment variable data, , is the system status dataset, consisting of The system status data set consists of system status. It's in time The first System status data; Calculate the change rate of environmental variables and system state parameters to evaluate the system's response speed to environmental fluctuations. The changes in different variables and state parameters are not just simple differences, but also need to consider the influence of time factors and nonlinear fluctuations. The calculation expression of the environmental change rate is as follows: , where yes The rate of environmental change within It's in time Moment The value of an environment variable indicates the value of the environment variable at the end of the detection window. It's in time Moment The value of an environment variable indicates the value of the environment variable at the end of the detection window. is an exponential decay factor used to control the The impact of environmental variables in the rate of change calculation; The calculation expression of the system state response change rate is as follows: , where yes The system state response change rate within It's in time Moment System status parameter value, indicating the system status parameter value at the end of the detection window. It's in time Moment System status parameter value, indicating the system status parameter value at the end of the detection window; To evaluate the direct impact of environmental changes on the system state, a coupling sensitivity formula is introduced, and the calculation expression is as follows: , where is the coupling sensitivity; Combined coupling sensitivity , System state response change rate and the rate of environmental change Generate the environmental interference sensitivity index, the calculation expression is as follows: , where is the environmental disturbance sensitivity index, is the balance coefficient, which is used to adjust the impact of the difference between environmental changes and system responses on the index. is a small constant that prevents the denominator from being zero, ensuring calculation stability.

Images