Intelligent monitoring system based on multi-sensor fusion technology

Abstract

The invention relates to the technical field of supervision control and data acquisition, in particular to an intelligent monitoring system based on a multi-sensor fusion technology, comprising a multi-sensor acquisition and preprocessing module which acquires and self-calibrates at least two kinds of physical quantity data in real time and fuses the physical quantity data into a unified monitoring feature vector; and the fault diagnosis and prediction module is used for integrating interpretable semantic analysis in combination with a production plan and a maintenance record. And performing equipment state diagnosis, operation trend analysis and fault prediction in real time by utilizing electric inspection type monitoring, generating health assessment and early warning, and guiding optimization and adjustment of a control strategy. And the data storage module is used for storing semantic data model evaluation results, diagnosis reports and early warning information, and supporting fault root analysis and control effect evaluation based on processing batches and time sequences. And the remote man-machine interaction interface is used for displaying the equipment running state, the control parameters and the automatic process in real time, supporting voice instruction interaction and facilitating remote monitoring and intervention of an operator.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of supervisory control and data acquisition, in particular to an intelligent monitoring system based on multi-sensor fusion technology. BACKGROUND

[0002] The existing industrial equipment monitoring system has significant defects in realizing remote fault monitoring and diagnosis. The traditional monitoring system relies on discrete data of single sensor and limited types of sensors, which is difficult to fully reflect the complex operation state of the equipment. For example, relying only on vibration data may not effectively reveal the thermal deformation failure caused by excessive temperature, and the limitation of single sensor data also makes it difficult to effectively deal with the problems of sensor drift and failure, resulting in inaccurate data and affecting the reliability of diagnosis.

[0003] Due to the lack of deep fusion capability of multi-source heterogeneous data, even if multiple data are collected, they are often simply superimposed, and the internal correlation between the data is not fully mined to form a unified and high-dimensional equipment health representation. Therefore, the existing supervisory control and data acquisition system often cannot effectively and accurately diagnose when facing early, complex and hidden faults, misses the best maintenance opportunity, causes equipment unplanned downtime, and causes production interruption and huge economic losses. The existing supervisory control and data acquisition system generally lacks advanced intelligent analysis capability in the aspect of fault diagnosis and prediction. The diagnosis logic is mostly based on preset rules and simple statistical models, which is difficult to adapt to the complexity and variability of industrial equipment operation conditions and the evolution of fault modes. The diagnosis result of the existing supervisory control and data acquisition system is often a black box output, lacking in explainability and transparency of the diagnosis conclusion, and the operator is difficult to understand the reasons and basis of the fault diagnosis given by the system, which directly leads to insufficient trust of the operator to the system, affecting the adoption of the diagnosis result and the efficiency of maintenance decision. In addition, the traditional system is relatively weak in data security and fault tolerance mechanism. When the key sensor fails, the system may face the risk of data interruption and even collapse, and cannot guarantee the continuity and reliability of monitoring.

[0004] Therefore, an intelligent monitoring system based on multi-sensor fusion technology is proposed. SUMMARY

[0005] The purpose of the present application is to realize intelligent monitoring of industrial equipment by fusing multi-sensor data and explainable artificial intelligence technology. Specifically, an intelligent monitoring system based on multi-sensor fusion technology is provided for remote fault monitoring and diagnosis of industrial equipment and production lines, comprising:

[0006] a multi-sensor acquisition and preprocessing module for acquiring at least two different types of physical quantity data in real time, calibrating the physical quantity data in real time using a self-calibration algorithm, and fusing the physical quantity data into a unified monitoring feature vector comprehensively representing the industrial equipment by using a Bayesian network;

[0007] a fault diagnosis and prediction module for quantifying the contribution of each input variable in the unified monitoring feature vector to the detected fault based on a deep incremental analysis algorithm integrating marginal contribution values, intelligently fusing the global semantics in combination with production plans and maintenance records, performing real-time industrial fault diagnosis, abnormal trend analysis, and residual life prediction using electrical inspection monitoring, and generating health state evaluation results and fault warning information of the industrial equipment;

[0008] a data storage module for storing the health state evaluation results, fault diagnosis reports, and warning information;

[0009] a remote human-computer interaction interface for displaying the real-time running state, historical fault data, diagnosis results, and warning information of the industrial equipment on site or remotely, and performing manual system configuration, alarm threshold setting, and remote diagnosis auxiliary operation.

[0010] Preferably, in an industrial processing scenario, the multi-sensor acquisition and preprocessing module acquires physical quantity data including temperature sensors for monitoring the running temperature of the industrial equipment and production line, force sensors for acquiring the processing load and cutting force of the industrial equipment, cutting force sensor data for monitoring the cutting force changes in the processing process, spindle vibration sensor data for detecting the vibration state of the processing equipment spindle, temperature sensor data for monitoring the thermal deformation of the processing tool and workpiece, vibration sensors for acquiring the mechanical vibration signals of the industrial equipment to provide basic vibration data for electrical inspection monitoring of the equipment, and a Bayesian network for converting the acquired physical quantity data into a unified monitoring feature vector representing the running health state of the processing equipment.

[0011] Extended Kalman filtering is used to calibrate the non-linear sensor model; a sliding window filter is used to dynamically adjust the size of the historical data window, balance real-time performance and calibration stability according to signal characteristics; historical reliability indicators of each sensor are used to dynamically allocate fusion weights; a spatiotemporal feature correlation analysis model is used to establish a multi-dimensional feature mapping relationship between sensors; multi-physical field coupling modeling is used to solve the cross-sensitivity problem of sensors; and a data-driven neural network is used to realize environmental parameter compensation.

[0012] Preferably, the multi-sensor acquisition and preprocessing module comprises fault-tolerant and safety mechanisms: a failed sensor fast isolation unit for real-time abnormal node rejection through consistency check of redundant sensor groups; an automatic switching to backup sensors and enabling virtual sensors based on digital twinning when critical sensors fail; a calibration parameter safety verification unit for setting physical logic boundary constraints on calibration parameters.

[0013] Preferably, the fault diagnosis and prediction module utilizes the deep incremental analysis algorithm to construct a feature contribution model and calculate Shapley values, quantifying the contribution of each input variable in the unified monitoring feature vector to the detected fault, monitors at least one of the following fault modes: mechanical component wear and imbalance; tool breakage and degradation; thermal anomalies leading to processing quality degradation; and generates a health state assessment result including fault probability, severity, and remaining life prediction;

[0014] The local fusion features in the unified monitoring feature vector are globally semantically aggregated, combined with external context information such as processing material attributes, process parameter settings, and environmental conditions; high-level self-supervised learning is performed to monitor potential fault signs in the processing process by training the model with unlabeled data; according to the results of global semantic aggregation and high-level self-supervised learning, the state assessment, abnormal diagnosis report, and predictive maintenance recommendations of the processing equipment are output.

[0015] Preferably, the fault diagnosis and prediction module further comprises a graph analysis function, specifically including: modeling the processing system of the industrial equipment as a graph structure, with nodes representing processing components and control units, and edges representing physical and control relationships between components; analyzing the propagation path of the fault signal, locating the fault starting point and impact range through graph traversal and shortest path algorithm; outputting a diagnosis report containing fault location, cause analysis, and recommended maintenance measures.

[0016] Preferably, the fault diagnosis and prediction module in the industrial processing scenario is configured to monitor at least one of the following processing equipment-specific fault modes based on the unified monitoring feature vector: tool wear and breakage; spindle bearing abnormalities; workpiece quality abnormalities caused by processing parameter deviations; and estimate the remaining useful life of key components of the processing equipment through a residual life prediction model, generating targeted fault warning information.

[0017] Preferably, in the industrial processing scenario, the data storage module includes: a local storage unit for caching real-time sensor data and processing state feature vectors collected during processing; using a semantic data model, storing semantic data of health state assessment results, historical fault diagnosis reports, and processing anomaly warning information of the processing equipment, and supporting fault data analysis based on processing batches and time series.

[0018] Preferably, in the industrial processing scene, the remote human-computer interaction interface is specially configured to: for various early warning and diagnosis suggestions, according to real-time monitoring data, diagnosis results and related historical records, display the real-time running parameters of the processing equipment, including cutting speed, spindle speed and processing load; provide processing failure trend analysis chart and historical processing quality data, expand detailed data, reasoning logic and historical cases through clicking and voice instructions, support operation and maintenance personnel to adjust processing parameters and replace tools and set processing equipment alarm threshold and diagnosis rules through remote operation.

[0019] Compared with the prior art, the beneficial effects of the present application are reflected in:

[0020] 1. Through multi-sensor fusion technology, it can collect and fuse various physical quantity data to form a comprehensive unified monitoring feature vector. Combined with deep incremental analysis algorithm, the system can not only diagnose faults, but also quantify the contribution of each input variable to the fault, greatly improving the accuracy and interpretability of the diagnosis results. This enables operation and maintenance personnel to better understand the root cause of the fault, rather than just identifying the fault phenomenon. For example, when detecting a decline in processing quality, the system can clearly indicate whether it is caused by abnormal cutting force, excessive spindle vibration or high tool temperature, and give specific contribution degree, thereby helping engineers quickly lock the problem source and avoid blind troubleshooting, greatly shortening the fault elimination time.

[0021] 2. The system integrates abnormal trend analysis and residual life prediction functions, which can predict potential failure risks and residual service life of key components in advance according to the health status evaluation results and historical data of the equipment. For example, by monitoring the vibration signal and temperature change of the spindle bearing, the system can give an early warning weeks or even months before the bearing fails, and estimate its residual life. This predictive maintenance mode replaces the traditional "after-the-fact maintenance" or "regular maintenance", avoiding the huge loss caused by sudden equipment downtime during production peak period, and avoiding unnecessary replacement of parts that have not been completely worn out, thereby effectively reducing maintenance cost, spare parts inventory and downtime, significantly improving equipment utilization and production efficiency.

[0022] 3. Through the remote human-computer interaction interface, the system supports the real-time running state, historical fault data, diagnosis result and early warning information of the on-site or remote display device. The operation and maintenance personnel can not only remotely monitor the device, but also perform manual system configuration, alarm threshold setting and remote diagnosis auxiliary operation. Especially in the industrial processing scene, the interface can display key parameters such as cutting speed, spindle speed and processing load, and provide a fault trend analysis chart, and even support clicking and voice instructions to expand detailed data and reasoning logic. This enables the operation and maintenance personnel to fully understand the device running condition and intervene without going to the scene, especially for widely distributed and large number of industrial devices, greatly improving the flexibility and efficiency of operation and maintenance, reducing the intensity and frequency of manual inspection, and optimizing the allocation of human resources. BRIEF DESCRIPTION OF DRAWINGS

[0023] Fig. 1 A flowchart of an intelligent monitoring system based on a multi-sensor fusion technology according to the present application is provided.

[0024] Fig. 2 A structural diagram of an intelligent monitoring system based on a multi-sensor fusion technology according to the present application is provided.

[0025] Fig. 3 A flowchart of an intelligent monitoring method based on a multi-sensor fusion technology according to the present application is provided. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0027] Please refer to Figs. 1 to 3 The present application provides an intelligent monitoring system based on a multi-sensor fusion technology, comprising:

[0028] A multi-sensor acquisition and preprocessing module: used for real-time acquisition of at least two different types of physical quantity data, real-time calibration of the physical quantity data by using a self-calibration algorithm, and fusion of the physical quantity data into a unified monitoring feature vector comprehensively representing an industrial device by using a Bayesian network;

[0029] Fault diagnosis and prediction module: integrate deep incremental analysis algorithm, quantify the contribution of each input variable in the unified monitoring feature vector to the detected fault, combine production plan, maintenance record for intelligent global semantic fusion, use electrical inspection type monitoring to conduct real-time industrial fault diagnosis, abnormal trend analysis and residual life prediction, generate health state evaluation results and fault warning information of the industrial equipment;

[0030] Data storage module: for storing the semantic data of health state evaluation results, fault diagnosis report and warning information;

[0031] Remote human-computer interaction interface: for displaying the real-time running state, historical fault data, diagnosis results and warning information of the industrial equipment on site or remotely, and performing manual system configuration, alarm threshold setting and remote diagnosis auxiliary operation.

[0032] As an embodiment of the present application, refer to Fig. 1 , a flowchart of an intelligent monitoring system based on multi-sensor fusion technology is proposed for the present application. Refer to Fig. 2 , a structural diagram of an intelligent monitoring system based on multi-sensor fusion technology is proposed for the present application.

[0033] Example one

[0034] This embodiment is to monitor the state of a five-axis CNC machining center when machining titanium alloy parts. The sensors deployed at key positions such as CNC spindle, tool holder and workbench, including force, vibration and temperature sensors, collect physical signals in real time, the edge computing node denoises and calibrates the original data, and extracts time domain and frequency domain features, and the edge node uses Bayesian network to fuse multiple source features into a comprehensive unified monitoring feature vector. As an embodiment of the present application, refer to Fig. 3 , a flowchart of an intelligent monitoring method based on multi-sensor fusion technology is proposed for the present application.

[0035] The feature vector is sent to the cloud server, and the deep learning model on the server conducts fault diagnosis and trend analysis. SHAP technology explains the model decision basis, and the graph analysis module locates the fault root. The system generates a report containing health score, fault type, confidence, residual useful life RUL and maintenance suggestion. The results and original features are stored in the database, and are presented to the operation and maintenance personnel in a visual way through the HMI interface, supporting remote configuration and intervention.

[0036] Two temperature sensors are deployed, one Pt100 platinum resistance temperature sensor close to the main shaft bearing seat to monitor the bearing operating temperature; one infrared temperature sensor aimed at the cutting area to monitor the tool-workpiece temperature; 1 force sensor, a three-way piezoelectric force gauge is installed under the workbench to collect cutting forces in three directions; 2 vibration sensors, two IEPE accelerometers are installed orthogonally on the spindle box to collect X, Y direction vibration signals. This is an important data source for electrical inspection monitoring, because vibration abnormalities often indicate mechanical failure, including imbalance, bearing wear and electrical failure; supplementary sensors include: acoustic emission sensors installed near the tool handle, which are extremely sensitive to the occurrence of small cracks in the tool. Spindle motor current sensor: monitor the current of the spindle drive motor through the Hall sensor, its fluctuation can reflect the load change and electrical abnormalities.

[0037] Further, the data acquisition uses NI cDAQ chassis and corresponding acquisition cards to synchronously collect force, vibration and AE signals at a sampling rate of 50 kHz, and to collect temperature signals at a sampling rate of 10 Hz.

[0038] Sensors will produce zero drift and sensitivity changes due to temperature, aging, etc. in long-term operation, and nonlinear phenomena are also common. Extended Kalman filter EKF linearizes the nonlinear problem by first-order Taylor expansion of the nonlinear function, and applies the Kalman filter framework for state estimation and calibration. Specifically, taking the force sensor as an example, its output may exhibit nonlinear drift due to temperature effects, and EKF will estimate the true force value in real time, thereby calibrating the error caused by temperature.

[0039] Further, the stationarity of the signal is judged by calculating its short-time energy and variance. When the variance is below the threshold, the window size is set to 2048 points; when the variance is above the threshold, the window size is reduced to 256 points.

[0040] A sensor health score (0-1) is maintained, which is based on the signal-to-noise ratio, drift rate of historical data, and consistency with redundant sensors. In fusion, the weight of the sensor is proportional to its health score.

[0041] The system first extracts time domain and frequency domain features from the preprocessed data collected by multiple sensors.

[0042] Further, a Bayesian network structure is constructed. In this network, nodes represent different physical quantity features, such as vibration root mean square, cutting force mean, and intermediate states such as bearing state, tool wear state, and finally converge into a top node, i.e. the overall health state of the device. The dependency relationship between nodes is given in the form of directed edges, which is determined by combining expert knowledge and data-driven algorithms.

[0043] Through the probabilistic inference of Bayesian networks, the system integrates these discrete, multi-dimensional sensor feature data, infers the posterior probability distribution of each intermediate state node, for example, the probability of tool wear given the vibration, force, and acoustic emission data. The posterior probability of these key state nodes, as well as some normalized original key features, collectively form the unified monitoring feature vector (UMFV).

[0044] This UMFV comprehensively represents the health status of industrial equipment, providing a unified and information-rich input for subsequent fault diagnosis and prediction. The system comprehensively collects key physical quantities such as temperature, force, cutting force, spindle vibration, etc., providing rich and detailed processing state information, and effectively identifying potential abnormalities. By using extended Kalman filtering, dynamic sliding window filtering, reliability-based fusion weight distribution, spatio-temporal feature correlation analysis, multi-physical field coupling modeling, and neural network environment compensation, the accuracy and reliability of the fused feature vector are greatly improved.

[0045] Further, the failed sensor rapid isolation unit is based on the consistency test of the redundant sensor group. Two IEPE accelerometers are installed orthogonally on the spindle box, which not only comprehensively captures the vibration in the X and Y directions, but also forms a redundant pair. The system continuously compares the correlation and amplitude difference of the signals of the two sensors. When the readings of a sensor, such as signal-to-noise ratio and drift rate, significantly deviate from the historical baseline of the other sensor, the system determines it as an abnormal node.

[0046] By comparing the temperature change trend of the Pt100 sensor close to the bearing seat and the infrared sensor aligned with the cutting area, combined with the processing conditions, a logical consistency test is performed. Once a main sensor is isolated, the system will automatically reduce its weight to zero and increase the weight of its redundant or functionally similar sensors to ensure uninterrupted data flow.

[0047] Further, in the extreme case of failure of a key sensor such as a force sensor and no physical backup, the system will activate a virtual sensor module based on digital twinning. This module uses the established equipment mechanism model and historical data to estimate the theoretical value of the failed sensor in real time according to the current working conditions such as spindle speed, feed rate, and motor current, as a temporary replacement to ensure the continuity of monitoring.

[0048] The calibration parameter safety verification unit aims to prevent incorrect calibration parameters from polluting the system and ensure the logical consistency of the physical world and the digital model.

[0049] When calibrating or setting parameters for sensors, the system has built-in verification rules. For example, when calibrating and configuring parameters for a three-way dynamometer, the system will strictly require that the cutting force gain coefficient must be positive; the temperature compensation coefficient must be within the pre-set range of material physical properties; in the non-cutting state and within a certain noise tolerance, the static reading of the force sensor after calibration should be close to zero; any parameter configuration that exceeds these pre-set boundaries will be rejected by the system and an alarm will be sent to the operator to prevent systematic bias caused by human error or malicious attacks.

[0050] The system can real-time eliminate faulty nodes and automatically switch to backup or digital twin virtual sensors to ensure key monitoring continuity, greatly enhancing the fault tolerance of the system; the calibration parameter safety verification unit can effectively prevent incorrect or malicious parameter settings through physical logic boundary constraints, ensuring the accuracy of data calibration and the overall safety of the system.

[0051] Further, an Attention-LSTM based on attention mechanism is used. LSTM is good at processing time series data UMFV sequence and can capture long-term dependencies. The attention mechanism allows the model to automatically focus on the most critical time points in the sequence when making a diagnosis.

[0052] Fault mode monitoring includes: (1) tool wear / breakage: characterized by smooth rise in cutting force, increase in high-frequency vibration energy, and sudden increase in AE signal; (2) abnormal spindle bearing: characterized by vibration peak at specific frequency of bearing fault and continuous rise in spindle temperature; (3) deviation of machining parameters: characterized by deviation of cutting force from normal range and abnormal vibration characteristics corresponding to workpiece surface quality.

[0053] Further, when the model predicts a "tool wear" probability of 92%, SHAP is immediately called to analyze and attribute the prediction result to each input feature, quantifying its contribution.

[0054] HMI displays a SHAP force diagram: [SHAP output]->prediction: tool is severely worn (probability: 0.92), high contribution features (red): cutting force = 480 N (+0.5), high-frequency vibration energy = 2.3 (+0.3), AE event number = 15 (+0.2), low contribution / suppression features (blue): spindle speed = 8000 rpm (-0.05).

[0055] The explanation is that "average cutting force" and "high-frequency vibration" are the two factors that most strongly prove that the tool is wearing out.

[0056] Further, the global semantic fusion and self-supervised learning splice the UMFV with external information from the MES system, such as workpiece material: Inconel 718, tool model: SEET13T3, program number: O2023, as extended input of the model, using a variational autoencoder VAE, trained on a large amount of unlabeled healthy condition data. The VAE learns the distribution of normal data. In real-time monitoring, if the reconstruction error of a new UMFV is much larger than the threshold, it is determined as an unknown abnormality and potential failure, even if it does not conform to any known failure mode.

[0057] Deep learning combined with SHAP deep incremental analysis algorithm can accurately quantify the contribution of each variable to the fault, making the diagnosis result transparent and persuasive, and facilitating precise root cause analysis. The system can effectively monitor multiple typical failure modes and provide comprehensive health assessment results including fault probability, severity and remaining useful life prediction. By aggregating external context information globally and combining high-level self-supervised learning, the system can deeply understand the semantics of failure and learn to discover potential failures from unlabeled data, improving the intelligence and universality of diagnosis.

[0058] Further, the main shaft motor, coupling, main shaft, front bearing, rear bearing, tool holder and tool are taken as nodes, and the construction logic of edges is that the motor drives the coupling, the coupling connects the main shaft, and the main shaft is supported by the front bearing and the rear bearing. The weight of the edge represents the connection stiffness and influence transmission coefficient.

[0059] The model diagnoses the main shaft anomaly, and SHAP shows that "front bearing characteristic frequency vibration" is the main cause. The system highlights the front bearing node on the graph and analyzes its adjacent nodes using graph traversal algorithms.

[0060] The output report is: "Fault location: front bearing. Possible causes: 1. Bearing wear and damage. 2. Source: Main shaft imbalance leading to bearing overload, affecting path: main shaft -> front bearing. 3. Source: Motor vibration transmitted through coupling, affecting path: motor -> coupling -> main shaft -> front bearing. Maintenance suggestion: First check the dynamic balance of the main shaft, then check the running state of the motor, and finally prepare to replace the front bearing."

[0061] Modeling the equipment machining system as a graph structure, the system can clearly analyze the fault signal propagation path and accurately locate the fault starting point and impact range through graph traversal and shortest path algorithms; the diagnosis report provides fault location, cause analysis and maintenance suggestions, which helps to develop more efficient and economical maintenance strategies from the system level.

[0062] The similarity-based remaining useful life RUL prediction method estimates the remaining life by matching the degradation trajectory of the current equipment with the degradation pattern of similar equipment in the historical database, with the core assumption that similar degradation behavior devices have similar RUL evolution trends.

[0063] Firstly, the degradation features of vibration, temperature, etc. are extracted and multi-dimensional trajectories are constructed. High-order features and interaction features are introduced to construct a unified monitoring feature vector. High-order features, such as the derivative or integral of the vibration signal, can capture the change trend or cumulative effect of the degradation rate.

[0064] Interaction features, such as the ratio or product of temperature and vibration amplitude, can reflect complex degradation phenomena under multi-physical field coupling. Then, the similarity between the current trajectory and historical cases is calculated using algorithms such as dynamic time warping, and the Top-K most matching historical degradation curves are selected. Finally, the prediction result is obtained through the RUL values of these similar cases. This RUL prediction method can more comprehensively understand the complex degradation mode of the equipment by extracting high-order and interaction features. The dynamic time warping algorithm is used to accurately match historical cases, ensuring that the prediction result is highly accurate and adaptable.

[0065] The complete life cycle UMFV sequence of the historical equipment is stored. When the new equipment is running, its current UMFV sequence is matched with the historical library, such as using the dynamic time warping (DTW) algorithm. The weighted average of the remaining life of the N most similar historical sequences is used as the RUL prediction value of the current equipment.

[0066] Output: "Tool remaining useful life (RUL) prediction: 3.5 hours. It is recommended to replace after the current batch is completed."

[0067] The system focuses on monitoring tool wear / breakage, spindle bearing abnormalities, and workpiece quality problems related to processing-specific faults caused by processing parameter deviations, has high industry applicability, and through the remaining life prediction model, the system can estimate the remaining useful life of key components of the processing equipment and generate targeted warnings to achieve on-demand maintenance, maximize component life, and avoid unexpected downtime.

[0068] A lightweight time series database InfluxDB is used. To ensure reliable transmission of data before writing to the database, a lightweight message queue is introduced as a front-end buffer.

[0069] 50kHz force, vibration, acoustic emission signals and 10Hz temperature signals collected from NI cDAQ are temporarily stored in raw or compressed format. When the network connection between the edge node and the cloud is temporarily interrupted, the data will not be lost and can be uploaded in batches after the connection is restored.

[0070] After diagnosing a specific fault, the maintenance personnel can retrieve high-fidelity raw data before and after the fault from the local area for more detailed offline analysis. A first-in, first-out rolling storage strategy is used. For example, only the raw data of the last 24 hours or the last processing batch is retained to control the storage space occupancy.

[0071] The edge computing node stores the highly condensed unified monitoring feature vector UMFV generated by the Bayesian network fusion in real time into local data, ensuring that these key analysis results are not lost.

[0072] The cloud storage unit is deployed on a cloud server, builds a semantic data model, uses a knowledge graph technology, uses an attribute graph model, and uses a graph database for storage and management. For massive historical time series data, a cloud-native time series database or object storage can be used for archiving, and the knowledge graph serves as an intelligent index on the upper layer. The storage content and the semantic model: in the cloud, data is no longer isolated numerical values, but is endowed with rich contextual information. This architecture can deeply associate fault data with batches, devices, etc., enabling efficient root cause analysis and pattern recognition. The knowledge graph as an intelligent index greatly improves the retrieval efficiency and decision support capability for massive historical data.

[0073] A fault diagnosis report is represented as an event node in the semantic model. The event node is connected to other entity nodes through directed edges based on processing batch fault data analysis: specifically, for example: "query all records of tool wear warning that occurred during the processing of titanium alloy TC4 batches, and compare the spindle vibration characteristics and cutting force averages at the time these records occurred."

[0074] In the knowledge graph, the target processing batch is first selected from the relevant data, and in these batches, records using TC4 titanium alloy as the material are further filtered out, then the warning events triggered during the processing of these TC4 materials are found. Finally, among these warning events, the records of the tool wear category are specifically extracted, and all related equipment, time, and feature data are retrieved.

[0075] Based on time series fault data analysis, specifically, "draw the remaining useful life RUL decay curve of the spindle bearing of No. 5 CNC machine tool in the past six months, and mark all the time points of tool replacement and cutting force overload warning on the graph." The system first retrieves RUL data from the time series database, then queries all the timestamps of tool replacement and maximum load events related to the machine tool spindle through the knowledge graph, and finally superimposes this information on the same graph to help the maintenance personnel find the correlation between RUL mutations and specific events. By superimposing RUL data, tool replacement, and force overload events on the same graph, the processing fault trend analysis graph can intuitively reveal the correlation between RUL changes and these key events, helping maintenance personnel quickly locate the cause of the mutation and perform more accurate fault warning and diagnosis.

[0076] The high-efficiency hierarchical data management supports local unit caching real-time data to ensure immediate response, and the cloud unit uses a semantic model to store health assessment and failure reports for structured analysis. It also supports deep analysis and process optimization: cloud storage supports failure data analysis based on processing batch and time series, which helps to find long-term trends and ultimately optimize processing parameters to improve product quality.

[0077] Integrated cockpit: CNC equipment list and its health status, 98% green; 75% yellow; less than 60% red; core parameters of selected equipment real-time dashboard, including spindle speed, load, feed rate, tool temperature and real-time vibration waveform / spectrum; latest warning information list, "ID-002 tool wear warning, severity: moderate, click to view details"

[0078] Further, click on the warning to enter the diagnosis details page: fault type is tool wear, occurrence time, confidence 92%, recommended measures "recommend replacing the tool within 3.5 hours", dynamically display the SHAP force diagram described above, display the equipment component diagram, highlight the fault path and root node; automatically call out the key features before and after the fault occurs, such as cutting force, vibration RMS time series chart, and compare with the normal baseline.

[0079] Further, provide historical fault query function, can be screened according to equipment, time, fault type; generate equipment OEE comprehensive efficiency report, fault Pareto chart, etc., help management decision-making. Remote adjustment of alarm threshold, such as vibration alarm limit, modification of diagnosis rules. Integrated voice recognition module, operation and maintenance personnel can query by voice: "query spindle temperature curve in the past week", the interface automatically jumps and displays the relevant information.

[0080] Remote human-computer interaction interface real-time display cutting speed, spindle speed, processing load and other key parameters, and provide fault trend chart and historical quality data, realize comprehensive monitoring; by clicking, voice command to expand detailed data, support operation and maintenance personnel to remotely adjust processing parameters, replace tool and set alarm threshold and diagnosis rules, greatly improve the ability of remote management and optimization.

[0081] This embodiment uses multi-sensor fusion and Bayesian networks to fuse multiple physical quantity data into a unified monitoring feature vector after self-calibration, comprehensively and accurately reflecting the health status of the equipment, effectively solving the limitations of a single data source. Second, the fault diagnosis and prediction module integrates a deep incremental analysis algorithm, which not only diagnoses faults and quantifies the contribution of each variable, but also intelligently integrates global semantics with production plans and maintenance records, significantly improving the accuracy of diagnosis and decision-making intelligence. The electrical inspection monitoring, abnormal trend analysis and remaining life prediction functions not only have monitoring and diagnosis functions, but also provide more detailed explanations of the underlying fault principles, including specific sensor selection, algorithm implementation, system architecture and interaction design. With the support of theory and simulation data, the system demonstrates how to extract value from raw physical signals and ultimately provide strong support for quality improvement and efficiency enhancement in industrial production in a transparent, reliable and action-guiding manner.

[0082] Embodiment Two

[0083] The application scenario of this embodiment is a large-scale automobile power assembly line with dozens of stations, including engine and gearbox assembly lines. The production line is highly automated, integrating robot arms, precision tightening equipment, fluid filling systems and online test benches. The stations are tightly coupled, and any single point failure can cause the entire line to stop production, causing significant losses. The implementation of this system in this scenario demonstrates its ability to handle heterogeneous devices, distributed data sources and complex causal chains.

[0084] In the assembly line scenario, the fault-tolerant mechanism goes beyond the redundancy of a single device and extends to the logical consistency check between stations and multiple physical fields. Specifically, a servo tightening gun is used to precisely tighten the engine cylinder head bolts at a certain station. The system simultaneously monitors the motor current sensor, torque sensor and angle encoder of the tightening gun. A healthy tightening process has a highly correlated and fixed pattern of current, torque and angle curves.

[0085] If the torque sensor shows that the target value has been reached, but the motor current is much lower than the historical normal value, or the angle encoder reading is abnormally small, the system will determine that at least one sensor is abnormal. It is not just to isolate the failed node, but to generate a "low sensor group credibility" alert to prompt physical inspection.

[0086] A robot arm is used to transport the gearbox shell between two stations with a fixed 30-second beat. If the temperature sensor used to monitor the gearbox of joint No. 3 of the robot arm fails.

[0087] The system cannot directly obtain the temperature, but it has all the context information of the robot arm running: motor current, motion trajectory, load, ambient temperature and historical running data. The system will activate a pre-trained thermodynamic model to estimate the theoretical temperature of joint 3 in real time according to the current working intensity and beat of the robot arm. This virtual value will temporarily replace the failed physical sensor to ensure uninterrupted monitoring of the risk of overheating of the robot arm and gain time for planned maintenance.

[0088] The fluid filling station fills the gearbox with lubricating oil, and the operator needs to set the filling parameters on the HMI, such as the target filling amount and the maximum flow rate. The calibration parameter safety verification unit in the background of the system is embedded with the process procedures of this vehicle model. For example, the standard oiling amount of a certain type of gearbox is 8.5±0.1 liters. When the operator tries to set the target filling amount to 9.5 liters, which may be a mistake or sets the maximum flow rate to a value that may damage the pipeline, the system will immediately reject the parameter input and highlight the standard range, forcing the operator to confirm or seek authorization from the supervisor. Preventing batch quality problems or equipment damage caused by parameter errors.

[0089] Further, sensors are deployed: motor current, vibration, temperature sensors are deployed at each joint of the robot arm; high-precision torque and angle sensors are integrated on the tightening device; installation pressure, flow, and liquid level sensors are integrated on the filling system; EOL: acoustic probes are integrated for listening to abnormal sounds, high-precision pressure sensors are integrated for leakage testing, and vibration sensors are integrated for imbalance testing. Industrial cameras and microphone arrays are deployed in the global area for quality monitoring and global abnormal sound recognition.

[0090] A distributed I / O architecture is adopted, and a high-speed acquisition module integrated with NI cDAQ is deployed at each key workstation or electrical cabinet.

[0091] All nodes are connected through industrial Ethernet and use the Precision Time Protocol (PTP) for microsecond-level time synchronization, ensuring accurate analysis of cross-workstation causal relationships.

[0092] Due to the complexity of the scene, a hierarchical fusion strategy is adopted. In the edge node of each workstation, a local Bayesian network is first constructed, and in the robot arm node, the vibration, current, and temperature features of the six joints are fused into intermediate states such as joint 1 health and joint 2 health.

[0093] Further, the health status of the assembly level, such as the health of each joint of the robot arm, the state of the gripper, and the station-level sensor features, such as the positioning accuracy identified by camera images, are fused again to generate a station-level unified monitoring feature vector Station-UMFV, such as the health of the tightening station, the efficiency of the handling station. In the cloud, the system combines all the Station-UMFV uploaded by the stations with the topology of the production line (station sequence) and the process flow (material flow) to construct a huge, dynamic production line health status atlas. Atlas nodes: represent equipment (robot arm A), components (joint 3), stations (tightening station), events (batch X), and failure modes (screw missing tightening).

[0094] Atlas edges: represent physical connections (A station -> B station), causal relationships (screw missing tightening -> EOL leak test failure), and subordinate relationships (joint 3 -> robot arm A).

[0095] Further, the cross-station fault diagnosis and root cause location report 3 product test failures for the EOL leak test station. The system immediately triggers an atlas query. Starting from the "leak test failure" event node, the reverse trace is performed along the causal relationship edges in the atlas. It is found that the "cylinder head tightening station" upstream has slight but synchronous abnormalities in the torque stability and final torque achievement rate features in the Station-UMFV when passing through the corresponding 3 products.

[0096] The system further drills down to the component level of this station and finds that the power supply voltage of the servo tightening gun has a slight fluctuation.

[0097] Diagnosis report: "The root cause of the EOL leak test failure is determined with a confidence of 95% to be due to the unstable power supply of the upstream cylinder head tightening station, resulting in insufficient bolt pretightening force. It is recommended to immediately check the power supply or voltage stabilizer of this station."

[0098] Decision support and visualization: The HMI interface is a digital twin dashboard of the entire line. Normal stations are displayed in green, stations with potential risks are displayed in yellow, and stations that have occurred faults or have serious performance degradation are displayed in red. The maintenance personnel can intuitively see the propagation chain of the fault, and according to the accurate suggestions given by the system, they can directly assign orders to the maintenance team of the corresponding station, greatly shortening the fault troubleshooting time.

[0099] This embodiment realizes the leap from single-point fault alarm to precise tracing and predictive maintenance of cross-station complex causal chain by constructing an intelligent diagnosis and decision support platform covering the whole production line, deeply integrating heterogeneous data sources such as machine arms and tightening equipment, and using layered fusion strategy and production line health status atlas. Not only can it actively avoid downtime and batch quality risks through virtual sensing, logical consistency checking and pre-emptive parameter error-proofing, but also can quickly locate the root cause when a fault occurs, greatly improving overall equipment efficiency and reducing operation and maintenance costs. Ultimately, through digital twin dashboards, the traditional production line is upgraded to a transparent factory with self-awareness, deep diagnosis and intelligent decision-making capabilities.

[0100] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and changes can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. An intelligent monitoring system based on multi-sensor fusion technology for remote fault monitoring and diagnosis of industrial equipment and production lines, characterized by, Comprise: A multi-sensor acquisition and preprocessing module for real-time acquisition of different types of physical quantity data, real-time calibration of the physical quantity data using a self-calibration algorithm, and fusion of the physical quantity data into a unified monitoring feature vector that comprehensively characterizes the industrial equipment using a Bayesian network; A fault diagnosis and prediction module that quantifies the contribution of each input variable in the unified monitoring feature vector to the detected fault based on a deep incremental analysis algorithm, integrates production plans and maintenance records for intelligent global semantic fusion, uses electrical inspection monitoring to perform real-time industrial fault diagnosis, anomaly trend analysis, and residual life prediction, and generates health state evaluation results and fault warning information for the industrial equipment; A data storage module for storing semantic data of health state evaluation results and fault warning information; A remote human-machine interface for displaying real-time running state, historical fault data, diagnosis results, and warning information of the industrial equipment on site and / or remotely, and performing manual system configuration, alarm threshold setting, and remote diagnosis auxiliary operation.

2. The intelligent monitoring system based on multi-sensor fusion technology as claimed in claim 1, wherein, In an industrial processing scenario, the multi-sensor acquisition and preprocessing module acquires physical quantity data including: Temperature sensors for monitoring the operating temperature of the industrial equipment and production line, and monitoring the thermal deformation of processing tools and workpieces; force sensors for acquiring the processing load and cutting force of the industrial equipment; cutting force sensor data for monitoring the cutting force changes during processing; spindle vibration sensors for detecting the vibration state of the processing equipment spindle; vibration sensors provide the basic vibration data for electrical inspection monitoring of the processing equipment; and a Bayesian network is used to convert the acquired physical quantity data into a unified monitoring feature vector that characterizes the running health state of the processing equipment; Extended Kalman filtering is used to process the calibration of nonlinear sensor models; a sliding window filter is used to dynamically adjust the size of the historical data window, balance real-time performance and calibration stability according to signal characteristics; based on the historical reliability index of each sensor, the fusion weight is dynamically allocated; a spatiotemporal feature correlation analysis model is used to establish a multi-dimensional feature mapping relationship between sensors; a multi-physical field coupling model is used to solve the cross-sensitivity problem of sensors; and a data-driven neural network is used to realize environmental parameter compensation.

3. The intelligent monitoring system based on multi-sensor fusion technology as claimed in claim 1, wherein, The multi-sensor acquisition and preprocessing module includes fault tolerance and safety mechanisms: A failed sensor rapid isolation unit for real-time abnormal node removal through consistency checking of a redundant sensor group; in the event of a key sensor failure, automatically switch to a backup sensor and enable virtual sensing based on digital twinning; A calibration parameter safety verification unit for setting physical logic boundary constraints on calibration parameters.

4. The intelligent monitoring system based on multi-sensor fusion technology as claimed in claim 1, wherein, The fault diagnosis and prediction module uses the deep incremental analysis algorithm to construct a feature contribution model and calculate Shapley values, quantifying the contribution of each input variable in the unified monitoring feature vector to the detected fault, monitoring at least one of the following fault modes: mechanical component wear and imbalance; processing tool breakage and degradation; thermal abnormalities leading to processing quality degradation; and generating health state evaluation results including fault probability, severity, and residual life prediction; The local fusion features in the unified monitoring feature vector are globally semantically aggregated, combined with external context information such as processing material attributes, process parameter settings and environmental conditions; high-level self-supervised learning is performed to train the model through unlabeled data to monitor potential failure signs in the processing process; according to the results of the global semantic aggregation and high-level self-supervised learning, the state evaluation, abnormal diagnosis report and predictive maintenance suggestion of the processing equipment are output.

5. The intelligent monitoring system based on multi-sensor fusion technology as claimed in claim 1, wherein, The fault diagnosis and prediction module further includes a graph analysis function, specifically including: The processing system of the industrial equipment is modeled as a graph structure, with nodes representing processing components and control units and edges representing physical and control relationships between components; through graph traversal and shortest path algorithms, the propagation path of the fault signal is analyzed to locate the fault starting point and the impact range; fault warning information containing fault location, cause analysis and recommended maintenance measures is output.

6. The intelligent monitoring system based on multi-sensor fusion technology as claimed in claim 1, wherein, The fault diagnosis and prediction module in the industrial processing scene is configured to monitor the fault modes specific to the processing equipment based on the unified monitoring feature vector: tool wear and breakage, main shaft bearing abnormalities, and workpiece quality abnormalities caused by processing parameter deviations; And through the residual life prediction model, the residual service life of the key components of the processing equipment is calculated to generate fault warning information.

7. The intelligent monitoring system based on multi-sensor fusion technology as claimed in claim 1, wherein, In the industrial processing scene, the data storage module includes: A local storage unit for caching real-time collected sensor data and unified monitoring feature vectors during processing; a cloud storage unit that uses a semantic data model to store semantic data of health state evaluation results, historical fault diagnosis reports and fault warning information of the processing equipment, and supports fault data analysis based on processing batches and time series.

8. The intelligent monitoring system based on multi-sensor fusion technology as claimed in claim 1, wherein, In the industrial processing scene, the remote human-computer interaction interface is specially configured to: For various warning and diagnosis suggestions, according to real-time monitoring data, historical fault diagnosis reports and fault warning information, display the real-time running parameters of the processing equipment, including cutting speed, spindle speed and processing load; Provide processing fault trend analysis charts and historical processing quality data, and through clicking and voice instructions, expand detailed data, reasoning logic and historical cases to support operation and maintenance personnel to adjust processing parameters and replace tools and set processing equipment alarm thresholds and diagnosis rules through remote operation.

Images