Multi-source sensing fusion equipment state real-time early warning method and system

Abstract

The invention relates to the technical field of artificial intelligence and industrial monitoring, discloses a multi-source sensing fusion equipment state real-time early warning method and system, and aims to solve the problems of shallow multi-source sensing data fusion level, poor early warning instantaneity, low sensitivity to weak fault signals and weak decision ability in existing equipment state monitoring. The method comprises the following steps: acquiring structural strain, a temperature field, a vibration spectrum, acoustic emission energy and micro-displacement data through a multi-source sensing array; and realizing millisecond-level clock synchronization and data normalization by using a time-space alignment engine. According to the scheme, full-coverage, high-precision and real-time state evaluation and graded early warning in the high-end equipment material testing process are achieved, the anomaly detection coverage rate is remarkably increased, the false alarm rate and the omission ratio are reduced, and the system robustness and the maintenance response efficiency under the complex working condition are enhanced.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and industrial monitoring technology, specifically relating to a method and system for real-time early warning of the status of multi-source sensor fusion equipment. Background Technology

[0002] With the deep integration of high-end equipment manufacturing and materials science, the role of materials testing and analysis in product development and quality control is becoming increasingly prominent. Various testing equipment monitors the response of materials under different environmental and load conditions, providing key data support for material performance evaluation.

[0003] In practical applications, the stability of equipment operation directly affects the accuracy and repeatability of test results. Especially under extreme conditions such as high temperature, high pressure, and high frequency alternating stress, even a small deviation in the state may lead to significant data deviation, thereby affecting the reliability of material judgment conclusions.

[0004] Among them, equipment condition monitoring technology, as a core link to ensure the credibility of the testing process, has gradually evolved from single parameter monitoring to multi-dimensional perception in recent years. By integrating multiple sensors such as temperature, vibration, strain, and acoustic emission, it can achieve a comprehensive characterization of the equipment's operating health status. Its basic goal is to identify abnormal signs in a timely manner and prevent sudden failures from interfering with the testing process.

[0005] Existing technologies suffer from the following shortcomings: First, the sensor data fusion layer is shallow, with most systems only achieving simple data layering or time synchronization, lacking effective integration of heterogeneous signals in the feature space and decision-making level, resulting in incomplete state representation. Second, the state warning model has poor real-time performance, relying on offline training and static threshold judgment, making it difficult to adapt to the dynamic degradation process of equipment and complex operating condition switching. Third, it lacks sensitivity to early weak fault signals, and existing methods are susceptible to noise interference, failing to effectively extract key features reflecting the deterioration trend of equipment. Finally, the system's autonomous decision-making capability is weak, with warning results mostly output in the form of alarm prompts, lacking a hierarchical judgment mechanism for fault type, severity, and development trend, making it difficult to provide accurate support for test task scheduling and maintenance intervention. These problems collectively restrict the intelligence level and practical effectiveness of state monitoring systems in high-end equipment material testing scenarios. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies by providing a multi-source sensor fusion method and system for real-time early warning of equipment status, which can effectively solve the problems in the background technology. Currently, in the material testing and analysis process in the field of high-end equipment manufacturing, the monitoring of equipment operating status generally relies on single sensor data or discrete sampling methods, which is difficult to comprehensively reflect the dynamic process of material performance evolution under complex working conditions. Due to the lack of an effective integration mechanism for multi-source heterogeneous sensor information, existing systems cannot achieve synchronous perception and collaborative analysis of multi-dimensional physical parameters such as stress, temperature, vibration, and deformation of key equipment components, resulting in lagging status assessment, low fault warning sensitivity, and high false alarm rate. Especially in extreme testing environments such as high temperature, high pressure, and strong impact, traditional monitoring methods are easily affected by noise interference and have difficulty capturing early weak abnormal signals, failing to meet the application requirements of high reliability and high precision material testing. This invention constructs a multi-level, closed-loop feedback-driven multi-source sensor fusion architecture, realizing dynamic modeling and real-time risk prediction of equipment operating status in the full time domain and full space dimension, significantly improving the accuracy, robustness, and timeliness of status monitoring during material testing.

[0007] To achieve the above objectives, the present invention provides the following technical solution: In one aspect, a real-time early warning system for the status of a multi-source sensor fusion device, comprising the following components: A multi-source sensor array module is used to collect multi-dimensional physical response signals generated by the equipment during material testing. The physical response signals at least cover structural strain, surface temperature field distribution, high-frequency vibration spectrum, acoustic emission energy, and local micro-displacement. The module consists of a distributed fiber optic grating sensor, an infrared thermal imaging unit, a triaxial accelerometer, a piezoelectric acoustic emission probe, and a laser displacement sensor. Each sensor is embedded in the key load-bearing area of ​​the equipment under test according to a preset spatial topology layout to achieve comprehensive capture of the mechanical behavior of the material. A time-space alignment engine for time synchronization and format normalization of raw sensor data. The engine completes millisecond-level clock synchronization based on the IEEE 1588 precision time protocol and converts heterogeneous sensor streams into standardized time tensor structures through a unified data middleware, ensuring that information with different sampling frequencies and data dimensions is aligned under a unified time-space reference. A multi-scale feature extraction and fusion unit is used to extract features of each sensing channel and establish cross-modal correlations. The unit uses a hierarchical convolutional neural network to process various sensing signals. Specifically, a deep residual network is used to extract transient impact features for vibration and acoustic emission signals, a two-dimensional dilated convolution is applied to temperature field images to capture thermal diffusion patterns, and a long-term dependency relationship is modeled for strain sequences through an expanded causal convolutional network. A state integration assessment model for generating a comprehensive health index of equipment status is used. The model receives a joint embedding vector from the output of a multi-scale feature fusion layer, dynamically allocates the contribution weights of each modality through an adaptive attention weighting mechanism, constructs a Bayesian state inference framework by combining historical degradation trajectories, and outputs a health score with continuous values ​​ranging from 0 to 1, representing the degree to which the current equipment deviates from normal operating conditions. A risk warning decision-maker is used to execute threshold determination and graded alarm strategies. The decision-maker sets three warning thresholds: when the health index is below 0.95, a level 1 alert is triggered and the trend tracking mode is started; when it is below 0.85, a level 2 warning is activated and local anomaly location information is pushed; when it is below 0.70, a level 3 emergency alarm is issued and a shutdown protection command is linked. A cloud-based collaborative management platform for storing historical data, updating model parameters, and supporting remote access. The platform has an edge-cloud dual-layer architecture, performs real-time inference operations at local edge nodes, and periodically uploads compressed feature fragments to the central server for global model iteration and optimization. It also supports configuration updates and diagnostic log downloads via a secure communication interface. Preferably, in the multi-source sensor array module, fiber optic grating sensors are arranged along the axial direction of the main load-bearing component at intervals not exceeding 5 cm. Each sensing point can simultaneously demodulate both strain and temperature information, with a wavelength resolution of ±1 pm, corresponding to a strain accuracy better than ±1 με and a temperature resolution of ±0.1℃. The infrared thermal imager has a spatial resolution of 640×480 pixels, a frame rate of 100 Hz, and covers the 3–5 μm mid-wave infrared band. Combined with a blackbody calibration source, it achieves a temperature measurement deviation within ±0.5℃. The triaxial accelerometer has a frequency response range of 0.1 Hz to 10 kHz and a sensitivity of 100 mV / g, used to capture vibration responses across the entire frequency range from slow creep to sudden impact.

[0008] Furthermore, the time-space alignment engine is equipped with a dynamic interpolation compensation algorithm. For sensing channels with a sampling rate lower than the overall reference frequency (such as temperature imaging), cubic spline interpolation is used to upsample them to a unified time grid, with a time step set to 1ms. For wireless nodes with transmission delays, a sliding window correlation matching method is introduced to estimate the delay offset, and a compensation timestamp is injected into the header of the data packet to ensure that all input signals are aligned to the sub-millisecond level before entering the fusion unit.

[0009] In addition, the multi-scale feature extraction and fusion unit includes a modality-specific preprocessing submodule: the vibration signal is first bandpass filtered (0.5kHz–8kHz) to remove DC drift and high-frequency noise, and then the short-time Fourier transform amplitude spectrum is calculated as the time-frequency input; the acoustic emission signal uses three engineering features—rise time, ring count, and energy integral—as auxiliary criteria; the strain sequence is denoised by wavelet and then input into the LSTM encoder to extract trend components; all modal feature vectors are mapped to a 128-dimensional unified latent space, and query-key value interaction is achieved through a cross-attention mechanism to generate a joint representation containing cross-modal semantic dependencies.

[0010] Preferably, the adaptive attention mechanism in the state integration evaluation model dynamically adjusts the weight coefficients of each sensor channel according to the variance level of the current input features. If the temperature gradient change in a certain area exceeds 20℃ / s or the cumulative acoustic emission energy increases by more than 3 times, the weight ratio of the corresponding channel is automatically increased to no less than 60%, thereby enhancing the response capability to sudden damage events. During the training phase of the model, a synthetic fault data augmentation strategy is adopted to simulate multiple failure modes such as crack propagation, connection loosening, and material fatigue, ensuring that the evaluation results cover typical working conditions.

[0011] Furthermore, the risk warning decision-maker integrates hysteresis comparison logic to prevent frequent alarm jitter caused by instantaneous disturbances: any warning level must be maintained for more than 5 seconds after being triggered to confirm its effectiveness; the release condition requires the health index to be higher than the original threshold +0.05 for 10 consecutive seconds before it can be downgraded; once an emergency stop command is issued, it must be manually reset and the system self-check must pass before it can resume operation to ensure operational safety.

[0012] In addition, the cloud-based collaborative management platform supports role-based access control mechanisms, allowing maintenance personnel to view multi-dimensional status dashboards in real time via terminal devices. These dashboards include features such as overlaying three-dimensional heatmaps to display structural hotspot distribution, comparative analysis of historical health curves, and timeline recording of early warning events. The platform also has a built-in model version management system, which allows for remotely pushing updated evaluation models to the edge, enabling software-defined monitoring strategy upgrades.

[0013] On the other hand, a method for real-time early warning of the status of a multi-source sensor fusion device includes the following specific steps: Step S110: The multi-source sensor array deployed in key parts of high-end equipment collects structural strain, surface temperature, vibration acceleration, acoustic emission signal and micro-displacement data in real time during the material testing process to form a multi-channel heterogeneous sensing flow. Step S120: Use the time-space alignment engine to perform high-precision clock synchronization and sampling rate normalization processing on all the accessed sensor data to generate a standardized time series dataset with a unified time reference. Step S130: Input the normalized modal data into the corresponding feature extraction sub-network to extract the key representation features in the time domain, frequency domain and spatial domain, and generate a joint feature vector through the cross-modal attention fusion module. Step S140: Input the joint feature vector into the state integrated evaluation model, combine it with historical operating data to calculate the current health score of the device, and output a continuous value in the range of 0 to 1; Step S150: Input the health score into the risk warning decision-maker, compare and judge according to the preset three-level threshold rule, and generate a warning signal of the corresponding level when the score is lower than the set threshold. Step S160: Push the warning result along with the abnormal location information to the cloud-based collaborative management platform, and execute prompts, records, or linkage protection actions according to the warning level.

[0014] Compared with the prior art, the present invention has the following beneficial effects: By constructing a sensor array containing multiple physical modes and implementing a refined spatial layout, full coverage monitoring of key areas of material testing equipment was achieved, overcoming the technical defects of large blind spots and poor representativeness in single-point measurement, and increasing the anomaly detection coverage rate to over 98%.

[0015] By introducing a deep learning-based multi-scale feature fusion mechanism, cross-modal semantic interaction is achieved while preserving the independence of each sensing modality. This significantly enhances the ability to identify complex damage (such as thermo-mechanical coupling fatigue), reducing the false alarm rate by 76% compared to the traditional threshold method and lowering the false negative rate to below 4%.

[0016] A health assessment model with dynamic weight allocation was designed, which can automatically adjust the contribution of different sensors according to the actual working conditions, improve the robustness of the system in non-steady-state and strong interference environments, and the health score response delay is less than 200ms, meeting the requirements for real-time early warning.

[0017] Adopting an edge-cloud collaborative architecture, it supports remote monitoring and model iteration while ensuring low-latency local decision-making, reducing operation and maintenance costs. It is suitable for clustered management scenarios of multiple sets of high-end equipment, with strong system scalability and a 50% improvement in deployment efficiency.

[0018] The early warning decision-making logic incorporates hysteresis control and manual confirmation mechanisms, which effectively suppresses erroneous actions caused by transient interference and improves the safety of human-machine collaboration. Combined with the multi-dimensional diagnostic information provided by the visualization platform, the fault tracing time is reduced to 1 / 3 of the original time, greatly improving maintenance response efficiency. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the structure of the real-time early warning system for the status of multi-source sensor fusion equipment proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of the multi-scale feature extraction and fusion unit in this invention. Detailed Implementation

[0020] Please refer to Figure 1 and Figure 2 The technical means and effects of the present invention to achieve the intended purpose will be further explained below with reference to the accompanying drawings and preferred embodiments. The specific implementation methods, structures, features and effects of the present invention will be described in detail below.

[0021] Example 1 In a high-end equipment materials fatigue testing platform, to address the operational status monitoring requirements of a certain type of aero-engine turbine disk under high-cycle fatigue loads, the multi-source sensor fusion equipment status real-time early warning system described in this invention was deployed. This testing condition is characterized by long cycles (over 200 hours of continuous operation per cycle), high load frequencies (excitation frequency up to 800Hz), and drastic environmental temperature changes (operating temperature range of 150℃ to 650℃). Traditional methods based on single vibration threshold alarms are insufficient to effectively identify early microcrack initiation and propagation trends, posing a significant risk of missed detections. Therefore, this embodiment constructs a closed-loop monitoring architecture integrating five types of physical modal sensing capabilities. Through multi-level data alignment, deep feature fusion, and dynamic health assessment mechanisms, it achieves high-precision modeling and millisecond-level response early warning of the entire lifecycle status evolution of key components.

[0022] First, a multi-source sensor array module, consisting of a distributed fiber optic grating sensor (FBG), an infrared thermal imaging unit, a triaxial accelerometer, a piezoelectric acoustic emission probe, and a laser displacement sensor, was deployed in stress concentration areas such as the rim, spoke transition zone, and tenon root of the turbine disk under test. The FBG sensor is embedded in the metal substrate surface at an average spacing of 4.7 cm along the main load-bearing path. Each sensing point has wavelength demodulation capabilities, enabling simultaneous acquisition of both strain and temperature information. Its wavelength resolution is ±0.9 pm, corresponding to a strain measurement accuracy of ±0.8 με and a temperature resolution of ±0.09℃, meeting the requirements for separating the thermal-mechanical coupling effect under high-temperature conditions. The infrared thermal imager is installed inside the protective cover, facing the radial section of the turbine disk, with a spatial resolution of 640×480 pixels, a frame rate of 100Hz, covering the 3–5 μm mid-wave infrared band, and is equipped with a blackbody calibration source for automatic radiometric calibration once per hour, ensuring that the surface temperature field measurement deviation is controlled within ±0.48℃. The triaxial accelerometer, fixed to the outer wall of the bearing housing, has a frequency response range of 0.1Hz to 10kHz and a sensitivity of 100mV / g. It is used to capture the full-spectrum dynamic response from low-frequency creep deformation to high-frequency impact vibration. The piezoelectric acoustic emission probe is attached to the edge of the disk, with a center frequency of 150kHz, a bandwidth of 50–400kHz, a preamplifier gain of 40dB, and a sampling rate of 5MS / s. It is specifically designed to detect elastic wave release events caused by microscopic defects within the material. The laser displacement sensor, employing the confocal principle, has a measurement range of ±2mm, a resolution of 50nm, and a focused spot diameter of less than 10μm. It is used for non-contact monitoring of local micro-displacement changes, and is particularly suitable for tracking the micro-oscillations of high-temperature rotating components.

[0023] The raw signals output by the aforementioned sensor types are connected to the local edge computing node via wired or wireless means, and then enter the time-space alignment engine to perform data warping operations under a unified spatiotemporal reference. This engine constructs a master-slave clock synchronization network based on the IEEE 1588 Precise Time Protocol (PTP). All sensor nodes are equipped with Ethernet interface cards supporting hardware timestamps. The master clock is provided with a UTC standard time reference by a GPS timing module, and the end-to-end clock synchronization error is stable within ±80μs. To address the issue of inconsistent raw sampling rates, a unified time grid time step of 1ms is set as the basic time unit for subsequent analysis. Specifically, the triaxial accelerometer samples at 10kHz and requires downsampling: first, it passes through an anti-aliasing low-pass filter (cutoff frequency of 4.8kHz), then it is downsampled to 1kHz using the zero-order hold method, and finally further compressed to a 1ms output granularity using a moving average window. The infrared thermal imager outputs image frame streams at only 100Hz, i.e., a complete temperature field matrix every 10ms, therefore requiring upsampling. This embodiment introduces a dynamic interpolation compensation algorithm, employing cubic spline interpolation to fill the temperature sequence with a 1ms time interval, generating a continuous pseudo-time series tensor. Simultaneously, addressing the data packet arrival delay issue caused by the use of a wireless transmission link in the piezoacoustic transmitting probe, a sliding window correlation matching mechanism is designed: two adjacent known synchronization channels (such as the FBG strain gauge and the Z-axis of the accelerometer) are selected as reference signals. A 50ms sliding window is constructed at the receiving end to calculate the position of the maximum cross-correlation between the signal to be corrected and the reference signal, thereby estimating the real-time transmission delay offset, typically fluctuating between 3 and 12ms. Subsequently, a compensated timestamp is injected into the data packet header, ensuring that all heterogeneous sensor streams achieve sub-millisecond alignment before entering the next processing stage.

[0024] The standardized time-series dataset, after spatiotemporal alignment, is organized in tensor form, with a dimensional structure defined as [T×M×D], where T represents the number of time steps (in 1 ms), M represents the number of modalities (M=5 in this example), and D represents the total number of spatial sampling points for each modality (e.g., FBG has 36 measurement points, the accelerometer has 3 channels, and the thermal imager is considered a single channel but contains 307,200 pixels). This tensor is fed as input into a multi-scale feature extraction and fusion unit, which contains a modality-specific preprocessing submodule and a hierarchical neural network architecture. The specific process is as follows: For vibration acceleration signals, a bandpass filter is first performed, with the passband range limited to 0.5kHz to 8kHz, to remove DC drift and high-frequency electromagnetic noise interference. Then, a short-time Fourier transform (STFT) is calculated for each 100ms window segment using a Hanning window function with a window length of 512 points and an overlap rate of 50%, resulting in a frequency domain amplitude spectrum as the time-frequency input feature map with dimensions [100, T×F], where F is the number of frequency bins (257 in this example). For acoustic emission signals, in addition to retaining the original waveform, three engineering feature parameters are extracted: rise time, ring-down count, and energy integral, reflecting the steepness, duration, and overall intensity of the signal, respectively, forming a three-dimensional auxiliary feature vector, which is updated using a sliding window with a period of 5ms. For strain sequences acquired by FBG, a 5-layer discrete wavelet decomposition using the Daubechies wavelet basis (db4) was performed to separate the trend and detail terms. After removing high-frequency noise components, the denoised signal was input into a Dilated Causal ConvNet, which contains 6 residual blocks with dilation factors of 1, 2, 4, 8, 16, and 32, covering a historical sequence of up to 192 ms, effectively modeling long-term dependencies. For infrared temperature field images, a 2D dilated convolutional layer was applied to extract the thermal diffusion pattern, with a hole ratio of 2 and a kernel size of 5×5, preserving a large range of spatial context information while avoiding excessive dimensionality reduction. For laser displacement signals, normalization was performed directly, and the signals were input into an LSTM encoder with a hidden layer dimension of 64, outputting a trend encoding vector.

[0025] After independent feature extraction for each modality, all feature vectors are mapped to a unified 128-dimensional latent space, with dimensional alignment achieved using a linear projection layer. The module then proceeds to a cross-modal attention fusion module, which constructs a query-key-value interaction structure based on a cross-attention mechanism. Vibration features are used as the query, and the concatenated features from the other four modalities serve as the key and value. An attention weight matrix is ​​calculated, and the final output is... for:

[0026] in, For query vector, For key vectors, For value vectors, For normalization function, Scaling factor For feature dimension, To swap the rows and columns of the key matrix so as to... Matrix multiplication is performed. This mechanism allows the system to identify semantic relationships between different modalities. For example, when a high-intensity acoustic emission event occurs at a certain moment, if it is accompanied by a sudden increase in local temperature and a sudden jump in micro-displacement, the attention weights will automatically enhance the energy contribution of specific frequency bands in the vibration signal during that period, thereby highlighting the composite anomaly characteristics. The fused joint feature vector still has a dimension of 128, representing the results of cross-modal collaborative sensing, and serves as the input to the state integration evaluation model in the next stage.

[0027] The state integration evaluation model receives the joint embedding vector and performs Bayesian state inference by combining it with the device's historical degradation trajectory. The model structure consists of two parts: an immediate inference branch and a memory tracking branch. The immediate inference branch comprises a two-layer fully connected network with the Swish activation function, outputting a preliminary health score for the current time step. The memory-tracking branch uses a gated circular unit (GRU) to maintain a hidden state. Record the health evolution path over the past 24 hours, and update it each time. Together with external covariates (such as cumulative cycle count and highest temperature history), the data is input into the GRU cell to generate the updated hidden state. Final health score Weighted by adaptive attention mechanism:

[0028] in The trend prediction value decoded from the hidden state. These are dynamic weighting coefficients, determined by the variance level of the current input features. If the temperature gradient change rate in a certain region exceeds 20℃ / s, or the cumulative acoustic emission energy increases more than threefold within 1 second, it is determined to be a sudden damage event. In this case, the weight ratio of the corresponding channel is increased to no less than 60%. A score of ≥0.6 ensures the model responds quickly to emergency situations. During the training phase, the model employs a synthetic failure data augmentation strategy, utilizing finite element simulation to generate a virtual sensing dataset containing 120,000 samples across various failure modes, including surface crack propagation, bolt loosening, and material oxidation degradation. Labels are generated as continuous health scores based on the remaining lifetime percentage. The loss function combines smoothed L1 loss and KL divergence, and the optimizer is AdamW. The initial learning rate is 3e-4, and the batch size is 256. After training convergence, the model is deployed to edge nodes.

[0029] Output health score The data is sent in real-time to the risk warning decision-making system, where a three-level threshold judgment logic is executed. The first-level warning threshold is set at 0.95. When the score is less than 0.95 and lasts for more than 5 seconds, a first-level alert is triggered. The system starts trend tracking mode, automatically increases the data sampling density to once every 0.5ms, and opens a high-frequency storage buffer to prepare to capture potential abnormal evolution processes. The second-level threshold is 0.85. Once it is exceeded and maintained for more than 5 seconds, a second-level warning is activated. The system uses a positioning algorithm to invert the spatial coordinates of the abnormal source. The specific method is: based on the time difference of arrival of the acoustic emission signal at each probe (TDOA), combined with the FBG strain peak distribution heat map and the infrared hot spot position, the least squares method is used to solve the three-dimensional spatial intersection point. The positioning accuracy can reach ±3mm. The position information, along with the multimodal feature comparison map, is pushed to the cloud platform. The third-level emergency alarm threshold is 0.70. When the score falls below this value, a third-level emergency alarm command is immediately issued, and the shutdown protection action is executed in conjunction with the action to cut off the drive power and release the braking device to prevent catastrophic breakage. All alert status transitions are subject to hysteresis comparison logic: after any level is confirmed to be in effect, the condition for de-escalation requires that the health index must be higher than the original threshold + 0.05 for 10 consecutive seconds before a downgrade can occur. For example, to recover from level two to level one, the following conditions must be met. >0.90 and maintain for more than 10 seconds. Once the emergency stop command is triggered, it must be manually reset by on-site maintenance personnel, and the test process can only be restarted after the system self-test program verifies that the communication of each sensor is normal and there are no residual abnormal characteristics, to ensure operational safety.

[0030] All early warning results, raw data fragments, feature vector snapshots, and diagnostic logs are uploaded to the cloud-based collaborative management platform via a secure communication interface. The platform employs a dual-layer edge-cloud architecture, with local edge nodes handling real-time inference and local alarm execution with latency below 200ms; the central server handles global data analysis, model iteration optimization, and remote access services. Uploaded data is encrypted with AES-256 and transmitted via TLS 1.3, stored in a distributed time-series database (InfluxDB cluster), and indexed by device ID, test batch, and timestamp. The platform includes a built-in model version management system, supporting A / B testing frameworks. New version evaluation models can be verified for performance improvements (e.g., a false alarm rate reduction of ≥15%) in an isolated environment before being remotely pushed to designated edge nodes via OTA for hot updates. Operations personnel can log in to the system via a web terminal to view a multi-dimensional status dashboard, including: a 3D heatmap overlay display of structural hotspot distribution (combined with CAD model rendering), historical health curve comparison analysis (supporting parallel display of multiple batches), and early warning event timeline records (including automatic attribution tags). In addition, the platform provides API interfaces for the MES system to call, enabling automated test task scheduling and early termination of invalid tests when the health score continues to deteriorate, thus saving energy and material costs.

[0031] This embodiment demonstrated excellent performance in actual operation, successfully capturing a microcrack initiation event occurring at the 187th hour: FBG data showed an abnormally high local strain growth rate (+12% / h), along with a sudden 2.8-fold increase in the acoustic emission energy integral value, but the total harmonic distortion rate did not change significantly, making it undetectable by traditional methods. This system detected the aforementioned multi-parameter coordinated anomaly through a cross-modal attention mechanism, rapidly reducing the health score from 0.97 to 0.82, triggering a secondary warning and accurately locating the crack at the root of the 12th tenon groove with an error of ±2.3mm. A shutdown inspection confirmed the presence of a surface-initiated crack approximately 0.4mm long, promptly preventing subsequent fracture. Statistical results show that this embodiment improved anomaly detection coverage to 98.7%, reduced the false alarm rate to 8.3%, and the missed detection rate to only 3.1%, with an average health score response delay of 187ms, fully meeting the high-reliability monitoring requirements of advanced materials testing scenarios.

[0032] Example 2 Unlike the end-to-end fusion architecture driven by deep learning in Example 1, this example proposes a hybrid modeling paradigm guided by physical mechanisms, focusing on addressing the problem of insufficient model generalization ability under conditions of limited fault samples. The application scenario is a novel titanium alloy sheet tensile testing machine, which operates in intermittent loading mode, with each test cycle lasting approximately 45 minutes. Having been in operation for less than a year, historical fault data is scarce, making it difficult to support effective training of a purely data-driven model. Therefore, this example reconstructs the system's technical roadmap, introducing explicit physical constraints and analytical model outputs in the feature extraction and fusion stages, forming a data + knowledge dual-driven fusion mechanism.

[0033] In this embodiment, the configuration of the multi-source sensor array module is basically the same, still including FBG, infrared thermal imager, triaxial accelerometer, acoustic emission probe and laser displacement sensor, but the layout strategy is adjusted: since the tensile specimen is a rectangular plate structure and the stress distribution has a central symmetry characteristic, the FBG sensor is arranged radially at the junction of the clamping end and the gauge length area, with a total of 24 measuring points and a spacing of about 6 cm; the infrared thermal imager is adjusted to an oblique side view to avoid strong reflection areas and improve the stability of surface temperature measurement; the triaxial accelerometer is installed at the bottom of the loading beam to mainly monitor lateral sway and torsional vibration; the acoustic emission probe is arranged in a ring array around the specimen, with a total of 4 probes, to improve positioning accuracy; the laser displacement sensor is changed to a dual-beam structure to measure the relative displacement difference between the upper and lower surfaces to calculate the flexural deformation of the specimen.

[0034] The temporal-spatial alignment engine maintains the same architecture, continuing to use the IEEE 1588 protocol for millisecond-level synchronization and employing a dynamic interpolation compensation algorithm to complete multi-rate data alignment. The key difference lies in the design logic of the multi-scale feature extraction and fusion unit. This embodiment no longer uses a unified deep neural network for end-to-end feature learning; instead, it divides feature extraction into two parallel paths: one data-driven path and the other model-driven path.

[0035] The data-driven approach retains some CNN / LSTM structures, but only for extracting uninterpretable nonlinear perturbation features. For example, vibration signals are still processed using STFT, and then input into a lightweight MobileNetV3-small network to extract spectral texture features; strain sequences are processed using 1D-CNN to extract local fluctuation patterns; and temperature field images are processed using shallow convolution to extract edge and patch features. These features together constitute the observation residual feature set, representing anomalous perturbation components that existing physical models fail to cover.

[0036] The model-driven approach constructs an explicit analytical model based on the principles of materials mechanics and heat transfer. Specifically, it includes: Elastic-plastic stress reconstruction model: Based on Hooke's law and the Ramberg-Osgood constitutive relation, combined with FBG measured strain data and load sensor readings, the internal stress distribution field of the specimen is inverted, as shown in the following formula:

[0037] in For elastic modulus, , , These are material parameters, determined by calibration tests. True stress represents the actual force per unit area within the material. To represent the true strain and the degree of material deformation, the model outputs a theoretical stress field. The residual is calculated by comparing it with the actual stress field estimated based on the strain gradient, and a stress deviation index is generated.

[0038] Heat conduction inversion model: Solved using the finite difference discretization of the Fourier heat conduction equation:

[0039] The input boundary condition is the surface temperature field measured by infrared. The initial conditions are room temperature and the material's thermal conductivity is... Specific heat capacity ,density Given parameters, This is the thermal diffusion term, representing the process of heat diffusion in space through conduction. Let be the rate of change of temperature over time, describing the dynamic evolution trend of temperature. The temperature field is represented by the implicit Euler method. The internal temperature distribution is solved iteratively, and the internal heat source term is estimated. If its value deviates significantly from zero If so, it can be determined that there may be abnormal energy release such as frictional heat generation or phase change heat release.

[0040] Vibration modal matching model: A simplified beam vibration model is established based on the specimen's geometric dimensions and material parameters, and the first five natural frequencies are calculated. to And perform peak matching with the measured acceleration spectrum. Define the modal offset rate. When any order exceeds 5%, it is marked as a sign of structural stiffness degradation.

[0041] The outputs of the three physical models—stress deviation, internal heat source intensity, and modal shift rate—are encoded into three mechanistic feature vectors with clear physical meaning. Subsequently, the system enters the fusion phase: instead of a simple attention mechanism, a Bayesian Belief Function Fusion Network is constructed. This network treats residual features from the data-driven path as uncertain observational evidence and mechanistic features from the model-driven path as strong prior evidence, synthesizing them according to Dempster-Shafer evidence theory.

[0042] in This represents the basic probability assignment function (BPA). The basic probability assignment function for data-driven paths, The basic probability assignment function for the physical model path. This follows the Dempster combination rule. For example, if the stress deviation in the physical model output reaches a critical value (>15%), a high confidence level is assigned to support the structural damage hypothesis; simultaneously, if the data-driven path detects enhanced acoustic emission activity, the composite confidence level of this hypothesis is further strengthened. The final output is a comprehensive confidence distribution, and the health level corresponding to the highest confidence hypothesis is selected as the intermediate result.

[0043] The intermediate result is input into an improved state integration assessment model, which no longer uses a pure neural network but instead employs a fuzzy inference system. Input variables include: overall confidence level, cumulative loading cycle count, maximum strain history value, and temperature cycle amplitude; the output is a health score. The membership function uses a Gaussian type, and the rule base contains 27 rules, for example: IF stress deviation is 'high' AND acoustic emission activity is 'intense' THEN health score is 'extremely low'. The inference method is Mamdani type, and the centroid method is used for defuzzification.

[0044] The risk warning decision-maker structure remains unchanged, still executing three-level threshold judgment and hysteresis logic, but its input health score comes from a different source and is more physically interpretable. For example, in one test, when the sample enters the yielding stage, the physical model correctly identifies the stress-strain nonlinear deviation but judges it as normal plastic behavior; while the data-driven path misjudges it as abnormal, which might trigger a false alarm if relying solely on deep learning; this embodiment suppresses this false alarm through an evidence fusion mechanism, and only when subsequent local necking leads to abnormal temperature concentration is a level-two warning truly triggered, demonstrating the robustness advantage brought by knowledge guidance.

[0045] The cloud-based collaborative management platform synchronously updates the strategy library, allowing experts to transform newly discovered failure mechanisms into new physical model modules and distribute them to the edge via OTA (Over-The-Air) updates, enabling the continuous evolution of monitoring capabilities. This embodiment is particularly suitable for the early monitoring stages of new materials or newly developed equipment, providing reliable condition assessments even in data-scarce situations, filling the application gap of purely data-driven methods.

[0046] Example 3 This embodiment focuses on the extended application of the system in a clustered management scenario of multiple sets of equipment, overcoming the limitations of Embodiments 1 and 2 which only addressed single-device applications. The application scenario is a general materials testing center at a new materials research institute, which has 12 high-end testing equipment of different types (tensile, compression, fatigue, creep) that need to be uniformly integrated into a status monitoring system. This embodiment reconstructs the system architecture, retaining the autonomous decision-making capabilities of each device's local edge nodes while constructing a centralized federated learning collaborative optimization framework. This enables cross-device knowledge sharing and model evolution, while ensuring data privacy and communication efficiency.

[0047] In this embodiment, each test device is equipped with a complete multi-source sensor array module and edge computing unit, independently running a local version of the early warning system, possessing complete S110 to S160 full-process processing capabilities. The difference lies in the upgrade of the cloud-based collaborative management platform to a federated learning coordination center, which no longer centrally stores raw data but instead periodically initiates global model aggregation tasks. The specific process is as follows: At the beginning of each month, the platform sends a model update request to all online devices; each edge node uses the latest month's accumulated running data (including normal and abnormal segments) to incrementally train its own state integration evaluation model, with the training objective being to minimize the local loss function. After optimization, upload the updated model parameters. (i.e., gradient vector or weight difference), rather than the original data.

[0048] The federated aggregation algorithm employs an improved FedAvg (Federated Averaging) strategy, introducing a device weight adjustment factor. The data quality score is dynamically weighted based on the historical data quality score of each device. The score is composed of three weighted indicators: sampling completeness (whether frames are missing), label accuracy (human verification confirmation rate), and operating condition diversity (load spectrum coverage), with a maximum score of 100. For example, a device that has been running standard aluminum alloy fatigue testing for a long time has a low diversity score due to its simple operating conditions. The lower weighting is used for testing multiple composite materials; while a single device capable of performing tests on various composite materials receives higher weighting. The polymerization formula is:

[0049] in, This represents the incremental update amount of the global model parameters. The total number of edge devices participating in the training. For device indexing, For the first Local model update volume of the device. For the first Weighting coefficients of each device, parameters of the new global model ,in, For the new global model parameters, These are the parameters of the old global model. The incremental updates to the global model parameters are securely verified, encapsulated into update packages, and sent one by one to each edge node via HTTPS. The entire process does not require uploading the original sensor data, complying with industrial data privacy protection requirements.

[0050] In addition, the platform has added an anomaly case sharing library function, using differential privacy technology to handle sensitive information. When a device triggers a level 3 alert and is confirmed as a genuine fault, maintenance personnel can choose to upload anonymized feature fragments (such as joint embedding vectors, data within 10 seconds before and after the inflection point of the health curve) to the sharing library, adding process tags (such as TC4 titanium alloy, high-temperature creep, grain boundary slip). Other devices can apply to download case features under similar operating conditions during local model training, adding them as regularization terms to the loss function to achieve cross-device transfer learning.

[0051] This embodiment also optimizes the resource scheduling mechanism: when a device is about to undergo a high-risk test (such as an ultra-high temperature ceramic fracture test), the platform can proactively push a recent successful early warning model of similar materials to the edge of the device, preload a backup model, and improve the reliability of initial monitoring. Experiments show that after adopting federated learning, the false alarm rate during the cold start phase of new devices decreased by 42%, and the overall system scalability was significantly enhanced, supporting future expansion to a cluster management of hundreds of devices.

[0052] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for real-time early warning of the status of a multi-source sensor fusion device, characterized in that, include: Multi-source sensor arrays deployed in key parts of the device under test are used to collect multi-dimensional physical response signals in real time during the material testing process. The multi-dimensional physical response signals include at least structural strain, surface temperature field distribution, high-frequency vibration spectrum, acoustic emission energy, and local micro-displacement. The time-space alignment engine is used to perform high-precision clock synchronization and sampling rate normalization on all the accessed sensor data, generating a standardized time series dataset with a unified time base. The normalized modal data are input into the corresponding feature extraction sub-network to extract their key representation features in the time domain, frequency domain and spatial domain, and a joint feature vector is generated through the cross-modal attention fusion module. The joint feature vector is input into the state integration evaluation model, and combined with historical operating data, the current health score of the equipment is calculated, and a continuous value representing the degree to which the equipment deviates from normal operating conditions is output. The health score is input into the risk warning decision-making unit, and the score is compared and judged according to the preset multi-level threshold rules. When the score is lower than the set threshold, a warning signal of the corresponding level is generated. The warning results, along with the abnormal location information, are pushed to the cloud-based collaborative management platform, and prompts, records, or linked protection actions are executed according to the warning level.

2. The method for real-time early warning of the status of a multi-source sensor fusion device according to claim 1, characterized in that, Multi-source sensor arrays deployed at key locations of the device under test are used to collect multi-dimensional physical response signals in real time during the material testing process, including: A sensor array consisting of a distributed fiber optic grating sensor, an infrared thermal imaging unit, a triaxial accelerometer, a piezoelectric acoustic emission probe, and a laser displacement sensor is embedded in the key load-bearing area of ​​the device under test according to a preset spatial topology layout, so as to simultaneously capture the all-round response of the material's mechanical behavior.

3. The method for real-time early warning of the status of a multi-source sensor fusion device according to claim 1, characterized in that, The time-space alignment engine performs high-precision clock synchronization and sampling rate normalization on all incoming sensor data, including: Millisecond-level clock synchronization of each sensor node is achieved based on a precision time protocol, and heterogeneous sensor streams are converted into standardized time-series tensor structures through a unified data middleware. For sensing channels with a sampling rate lower than the overall reference frequency, a dynamic interpolation compensation algorithm is used to upsample them to a unified time grid. For wireless sensor nodes with transmission delays, a sliding window correlation matching method is introduced to estimate the delay offset, and a compensation timestamp is injected into the packet header.

4. The method for real-time early warning of the status of a multi-source sensor fusion device according to claim 1, characterized in that, The normalized modal data are input into the corresponding feature extraction subnetwork to extract their key representation features in the time, frequency, and spatial domains, including: After bandpass filtering the vibration signal, the short-time Fourier transform amplitude spectrum is calculated and used as the time-frequency input; The rise time, ring count, and energy integral of the acoustic emission signal are extracted as engineering features. After wavelet denoising of the strain sequence, the trend component is extracted by inputting it into a long short-term memory network encoder. A two-dimensional hollow convolutional layer is applied to the temperature field image to capture the thermal diffusion pattern.

5. The method for real-time early warning of the status of a multi-source sensor fusion device according to claim 1, characterized in that, A joint feature vector is generated through a cross-modal attention fusion module, including: The feature vectors extracted from each modality are mapped to a latent space of a unified dimension; Using any modal feature as the query, the remaining modal features are concatenated as the key and value. Weights are calculated through a cross-attention mechanism to generate a joint representation containing cross-modal semantic dependencies.

6. The method for real-time early warning of the status of a multi-source sensor fusion device according to claim 1, characterized in that, The joint feature vector is input into the state integration evaluation model, and combined with historical operating data, the current device's health score is calculated, including: The joint feature vector is received and the contribution weights of each modality are dynamically allocated via an adaptive attention weighting mechanism. A Bayesian state inference framework is constructed by combining the historical degradation trajectory of the equipment, and the output is a continuous health score between zero and one.

7. The real-time early warning method for the status of a multi-source sensor fusion device according to claim 6, characterized in that, The adaptive attention weighting mechanism dynamically adjusts the weight coefficients of each sensor channel based on the variance level of the current input features. When a sudden damage event feature is detected, the weight ratio of the corresponding channel is automatically increased.

8. The method for real-time early warning of the status of a multi-source sensor fusion device according to claim 1, characterized in that, The health score is input into the risk warning decision-making device, and comparison and judgment are performed according to preset multi-level threshold rules, including: Set at least three warning thresholds to correspond to warning signals of different severity levels; The integrated hysteresis comparison logic requires that any warning level must be maintained for more than a preset time after being triggered before it is confirmed to be effective. The cancellation condition requires that the health index be continuously higher than the original threshold plus an offset for another preset time. Once an emergency stop command is issued, the system can only resume operation after a manual reset and a successful system self-test.

9. A real-time early warning system for the status of a multi-source sensor fusion device, characterized in that, include: A multi-source sensor array module is used to collect multi-dimensional physical response signals generated by the equipment during material testing. The time-space alignment engine is used to perform time synchronization and format normalization processing on raw sensor data; A multi-scale feature extraction and fusion unit is used to extract features from each sensing channel and establish cross-modal correlations. A condition-integrated assessment model is used to generate a comprehensive health index for equipment conditions. Risk warning decision-maker, used to execute threshold determination and graded alarm strategies; A cloud-based collaborative management platform is used to store historical data, update model parameters, and support remote access.

10. The real-time early warning system for the status of a multi-source sensor fusion device according to claim 9, characterized in that, The cloud-based collaborative management platform features an edge-cloud dual-layer architecture, performing real-time inference operations at local edge nodes while periodically uploading compressed feature fragments to the central server for global model iteration and optimization. It also supports configuration updates and diagnostic log downloads via a secure communication interface. The multi-scale feature extraction and fusion unit uses a hierarchical neural network to process various sensor signals and achieves feature fusion through a cross-modal attention mechanism. The state integration evaluation model combines historical degradation trajectories to construct a Bayesian inference framework to output a continuous health score.

Images