A bridge structure multi-source information collaborative monitoring and real-time diagnosis method based on edge computing

Abstract

This invention discloses a method for collaborative monitoring and real-time diagnosis of bridge structures based on edge computing, comprising: constructing a collaborative three-layer architecture of "end-edge-cloud"; collecting multi-source monitoring data at the perception layer; performing data preprocessing and adaptive fusion of multi-scale data at edge nodes to obtain structural state feature vectors; real-time diagnosis based on a lightweight model, outputting damage probability and uncertainty; determining whether to offload tasks to the cloud platform according to a dynamic task offloading strategy; deep analysis of complex models by the cloud platform; visualization of structural state based on a digital twin model; triggering early warning response according to a hierarchical early warning threshold system; and continuous optimization of the model by the cloud platform and its distribution to edge nodes. This invention achieves real-time and accurate diagnosis and hierarchical early warning of bridge structural state, with advantages such as low diagnostic latency, small data transmission volume, high diagnostic accuracy, and strong system reliability.

Description

Technical Field

[0001] This invention belongs to the field of bridge structure health monitoring technology, specifically relating to a method for collaborative monitoring and real-time diagnosis of bridge structures based on edge computing multi-source information. Background Technology

[0002] As a key node in transportation infrastructure, the structural safety of bridges directly affects the safety of people's lives and property and the operation of the national economy. With the rapid development of transportation infrastructure construction in my country, the number of bridges in service continues to grow, with the proportion of bridges that have been in operation for more than 20 years increasing year by year, and problems such as structural aging and damage accumulation becoming increasingly prominent. Traditional bridge structural health monitoring mainly relies on regular manual inspections and simple sensor data collection and analysis, which has the following prominent problems: First, the inspection frequency is low and the timeliness is poor, making it difficult to capture sudden damage; second, the data transmission and processing latency is large, and under the traditional cloud center architecture, the massive amount of monitoring data transmitted to the cloud for processing leads to response delays; third, it is difficult to integrate multi-source heterogeneous data, and parameters such as stress, vibration, displacement, and environment are difficult to analyze collaboratively; fourth, the diagnostic model has weak generalization ability and is not sensitive to early damage identification.

[0003] In recent years, researchers have begun exploring the application of edge computing and artificial intelligence technologies in bridge structural health monitoring. Edge computing, by deploying computing and storage resources at the network edge close to the data source, enables localized data processing and real-time analysis, effectively addressing the latency, bandwidth, and security issues of traditional cloud center architectures. However, existing technologies still face the following technical bottlenecks:

[0004] First, edge computing nodes are resource-constrained, making it difficult to complete complex data fusion and diagnostic tasks under low power conditions. Typical edge nodes use ARM architecture processors, which have limited computing power, while deep learning models (such as ResNet-50) require billions of floating-point operations for a single inference, and traditional CPUs cannot meet the performance and power consumption constraints of edge devices.

[0005] Second, the differences in temporal scale, spatial resolution, and physical significance among multi-source heterogeneous monitoring data make data fusion difficult. Studies have shown that structural response data exhibits significant multi-scale characteristics: temperature changes slowly (sampling frequency 0.1 Hz), structural vibration changes rapidly (sampling frequency up to 1000 Hz), and stress changes caused by vehicle loads fall somewhere in between. This multi-scale characteristic presents a challenge to data fusion.

[0006] Third, edge diagnostic models have limited accuracy and generalization ability, making it difficult to adapt to different bridge structures and environmental conditions. A single model struggles to accurately identify structural damage from massive amounts of monitoring data and lacks sufficient noise resistance.

[0007] Fourth, the collaboration mechanism between edge nodes and the cloud platform is imperfect, making it impossible to achieve optimized allocation of diagnostic tasks and collaborative fusion of diagnostic results. Existing task offloading algorithms suffer from problems such as mutual interference between agents and policy degradation when dealing with large-scale edge computing systems.

[0008] Therefore, developing a method that can fully utilize the advantages of edge computing to achieve collaborative perception, real-time diagnosis, and intelligent early warning of multi-source information for bridge structures has significant theoretical and engineering application value. Summary of the Invention

[0009] I. Purpose of the Invention

[0010] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for collaborative monitoring and real-time diagnosis of bridge structures based on edge computing. By constructing a three-layer architecture of "end-edge-cloud" collaboration, designing an FPGA-based adaptive edge intelligent node and a multi-scale data fusion algorithm, and establishing an edge-cloud collaborative diagnosis mechanism, the invention achieves real-time accurate diagnosis and graded early warning of the bridge structure status.

[0011] II. Technical Solution

[0012] To achieve the above objectives, the present invention provides the following technical solution:

[0013] A method for collaborative monitoring and real-time diagnosis of bridge structures based on edge computing multi-source information, characterized by the following steps:

[0014] Step 1: Construct a three-layer monitoring and diagnostic system architecture that is "device-edge-cloud" collaborative, including a perception layer, an edge layer, and a cloud platform layer; the perception layer consists of a collaborative perception network composed of various intelligent sensors deployed in key parts of the bridge structure; the edge layer consists of edge computing nodes deployed on the bridge site, with each edge computing node connected to multiple intelligent sensors; the cloud platform layer is a remote cloud computing platform that connects multiple edge computing nodes.

[0015] Step 2: The intelligent sensors in the perception layer collect multi-source monitoring data of the bridge structure and send the collected data to the corresponding edge computing nodes; the multi-source monitoring data includes stress-strain data, vibration data, displacement data, environmental load data, and surface defect image data;

[0016] Step 3: The edge computing node preprocesses the received multi-source monitoring data, including data cleaning, outlier removal, missing value compensation, and time synchronization, to obtain preprocessed multi-source monitoring data;

[0017] Step 4: Edge computing nodes perform multi-scale adaptive data fusion on the preprocessed multi-source monitoring data to obtain structural state feature vectors; the multi-scale adaptive data fusion includes three levels: data-level fusion, feature-level fusion, and decision-level fusion.

[0018] Step 5: The edge computing node performs real-time diagnosis on the structural state feature vector based on the lightweight diagnostic model and outputs preliminary diagnostic results; the preliminary diagnostic results include structural state category, damage probability and diagnostic uncertainty estimate;

[0019] Step 6: Based on the preliminary diagnosis results and the dynamic task offloading strategy, the edge computing node determines whether the diagnosis task needs to be offloaded to the cloud platform. If so, the original monitoring data, structural state feature vector, and preliminary diagnosis results are uploaded to the cloud platform to trigger in-depth cloud diagnosis. Otherwise, only the preliminary diagnosis results are uploaded to the cloud platform.

[0020] Step 7: The cloud platform receives the data uploaded by the edge computing nodes, performs in-depth analysis based on a complex diagnostic model, and outputs in-depth diagnostic results; the in-depth diagnostic results include damage location, damage type, damage degree, and structural safety margin.

[0021] Step 8: The cloud platform maps the deep diagnostic results to the digital twin model of the bridge structure based on the digital twin model, so as to realize the visualization of the structural status.

[0022] Step 9: Based on the preliminary diagnosis results and the in-depth diagnosis results, the cloud platform uses a decision-level fusion method to fuse the results, obtain the final diagnosis conclusion, and trigger the corresponding early warning response according to the preset hierarchical early warning threshold system;

[0023] Step 10: The cloud platform uses the aggregated multi-bridge data to continuously train and optimize the diagnostic model, and regularly distributes the updated lightweight model to edge computing nodes to achieve continuous evolution of diagnostic capabilities.

[0024] Furthermore, the intelligent sensor mentioned in step 1 includes a fiber optic grating sensor, a MEMS accelerometer, a nanoscale strain gauge, a BeiDou / GNSS receiver, an environmental sensor, and a vision sensor; the intelligent sensor has an adaptive sampling frequency adjustment function, which can dynamically adjust the data acquisition frequency according to the instructions of the edge computing node.

[0025] Furthermore, the edge computing node in step 1 adopts an FPGA+ARM heterogeneous computing architecture, where the FPGA is responsible for data parallel processing tasks, including signal filtering, feature extraction, and lightweight neural network inference; the ARM is responsible for task scheduling, communication management, and protocol conversion; the edge computing node supports dynamic partial reconfiguration of FPGA logic functions and can dynamically load different hardware acceleration modules according to the monitoring task requirements.

[0026] Furthermore, the data preprocessing described in step 3 specifically includes the following sub-steps:

[0027] Step 3.1: Data Cleaning: Identify and remove abnormal data points caused by sensor malfunctions or communication errors, using... The principle for identifying outliers is to determine that a data point is an outlier and remove it when it meets the following condition:

[0028]

[0029] in, For the current data point, The mean within the sliding window. This represents the standard deviation within the sliding window.

[0030] Step 3.2: Missing Value Compensation: For transiently missing data, linear interpolation is used for compensation. For continuously missing data, a prediction method based on Gaussian process regression is used for compensation. The Gaussian process regression model can be expressed as:

[0031]

[0032] in, It is a mean function. The covariance function (kernel function) uses a squared exponential kernel:

[0033]

[0034] Step 3.3: Time synchronization: Based on the IEEE 1588 Precision Time Protocol (PTP), all sensor data are synchronized to a unified time reference with a synchronization accuracy of microseconds.

[0035] Furthermore, the data-level fusion in step 4 specifically includes: fusing multi-sensor measurements of the same physical quantity using an improved Kalman filter algorithm to obtain the optimal estimate; the state-space model of the improved Kalman filter algorithm is as follows:

[0036] Equations of state:

[0037]

[0038] Observation equation:

[0039]

[0040] in, Let be the system state vector. Here is the state transition matrix. To control the input, For the control matrix, For process noise, The process noise covariance matrix is... For the observation vector, For the observation matrix, To observe the noise, To observe the noise covariance matrix.

[0041] The prediction steps of Kalman filtering are as follows:

[0042]

[0043]

[0044] The update steps are as follows:

[0045]

[0046]

[0047]

[0048] The improved Kalman filter algorithm employs an adaptive covariance matching method, dynamically optimizing the fusion weights based on sensor noise characteristics and structural state changes, i.e., real-time estimation of the observation noise covariance matrix. :

[0049]

[0050] in, For the innovation vector, Forgetting factor, Forgetting factor constant ( ).

[0051] Furthermore, the feature-level fusion described in step 4 specifically includes the following sub-steps:

[0052] Step 4.1: Synchronize multi-source monitoring data with different sampling frequencies to a unified time base using interpolation or resampling methods. For high-frequency data (such as vibration data), extract statistical features within the time window; for low-frequency data (such as temperature data), use linear interpolation to generate a time series synchronized with the high-frequency data. Time window length. Adaptive adjustment based on structural dynamic characteristics:

[0053]

[0054] in, The highest structural modal frequency of interest. This is an empirical coefficient (usually taken as 2~5).

[0055] Step 4.2: Extract physical and statistical features from each type of data. For vibration data, the improved fully adaptive noise ensemble empirical mode decomposition (ICEEMDAN) algorithm is used to decompose the data and obtain the intrinsic mode function (IMF) components:

[0056]

[0057] in, For the first Each intrinsic mode function component For the residual components.

[0058] Calculate the Hurst index for each IMF component and filter out components sensitive to damage:

[0059]

[0060] in, For recalibrated range, The time series length is used as the filtering criteria. , This is an empirical threshold (usually taken as 0.5).

[0061] Feature parameters are extracted from the filtered components: intrinsic frequencies. Damping ratio , vibration shape By performing modal parameter identification on the vibration transfer function, the following can be obtained:

[0062]

[0063] in, For the system poles, This is a residue.

[0064] For strain data, extract the stress level. Stress amplitude Stress cycle counting Fatigue-related characteristics are achieved based on the rainflow counting method:

[0065]

[0066] For image data, extract the crack length. ,width ,area Based on defects and other features, pixel-level segmentation is achieved using a convolutional neural network.

[0067] Step 4.3: A deep neural network is used to fuse the extracted multi-source features. A multi-input neural network architecture is designed, with each type of feature input into an independent sub-network for feature transformation. Let the first... The subnetwork output of class features is An attention-weighted fusion mechanism is used in the fusion layer:

[0068]

[0069] in, The projection matrix maps each feature vector to a uniform dimension. , Attention weights are calculated as follows:

[0070]

[0071] in, , , These are learnable parameters.

[0072] Furthermore, the decision-level fusion described in step 4 specifically includes: when multiple diagnostic models or multiple edge computing nodes respectively provide structural state assessment results, fusion is performed using DS evidence theory. Let the identification framework... For the set of structural state categories, the first The basic probability allocation (BPA) of each piece of evidence is: ,satisfy:

[0073]

[0074] Multiple pieces of evidence were fused using Dempster's synthesis rules:

[0075]

[0076] Among them, conflict factors .

[0077] Furthermore, the lightweight diagnostic model described in step 5 is constructed using depthwise separable convolution, channel shuffling, or bottleneck structure techniques. For one-dimensional time series data, a hybrid architecture based on convolutional neural networks and bidirectional long short-term memory networks (CNN-BiLSTM) is adopted:

[0078]

[0079] in, This is the hidden state of the forward LSTM. This is the hidden state of the backward LSTM.

[0080] Model compression uses 8-bit fixed-point quantization (INT8) instead of 32-bit floating-point quantization (FP32). The quantization process is as follows:

[0081]

[0082] in, For floating-point values, The quantized integer value. Scaling factor Zero-point offset, For the number of quantization bits (this patent takes) ).

[0083] Furthermore, the dynamic task offloading strategy described in step 6 describes the large-scale multi-access edge computing task offloading problem as a partially observable Markov decision process (POMDP). Define the state space. Observation space Action space and reward function In the time slot Edge nodes The observation vector is Decision network output actions Get immediate rewards .

[0084] The mean-field multi-agent reinforcement learning algorithm is used to reduce the dimensionality of the joint action space, and the Q-value function is approximated as:

[0085]

[0086] in, This is an approximate action for the mean field.

[0087] Dynamic task unloading strategy based on damage probability and diagnostic uncertainty The indicators specifically include:

[0088] when and At this time, the edge computing nodes complete the diagnosis independently, and only the preliminary diagnosis results are uploaded to the cloud platform;

[0089] when or When cloud-assisted diagnosis is triggered, the edge computing node uploads the raw monitoring data and feature vectors to the cloud platform;

[0090] when At that time, cloud-based in-depth diagnostics are triggered, edge computing nodes upload detailed data, and the cloud platform initiates an in-depth analysis process.

[0091] in, , , As a preset threshold, this patent takes , , .

[0092] Furthermore, the complex diagnostic model described in step 7 includes a crack identification model based on ResNet-50, a damage identification model based on ICEEMDAN-CNN, and a parameter inversion model based on finite element model correction. The loss function of the ICEEMDAN-CNN model is:

[0093]

[0094] in, For cross-entropy loss, For L2 regularization terms, This is the regularization coefficient.

[0095] Furthermore, the digital twin model described in step 8 constructs a three-dimensional geometric model of the bridge based on BIM and GIS technologies, and integrates a finite element model based on state-space equations on top of the geometric model. The structural dynamics equations can be expressed as:

[0096]

[0097] in, For the quality matrix, Here is the damping matrix. Here is the stiffness matrix. For displacement vectors, This is the external load vector.

[0098] Transformed into state-space equation form:

[0099]

[0100]

[0101] Wherein, the state vector System matrix:

[0102]

[0103] The Guyan reduction method is used to reduce the model order, retaining the displacement response of the main degrees of freedom (monitoring points) and consolidating the secondary degrees of freedom. The reduced stiffness matrix is:

[0104]

[0105] Among them, subscript Indicates the main degree of freedom. Indicates the secondary degree of freedom.

[0106] Real-time monitoring data is mapped to corresponding locations in the digital twin model, and model parameters are corrected through model update algorithms to make the model response approximate the measured response.

[0107]

[0108] in, For parameters to be corrected (such as elastic modulus, boundary conditions, etc.). The response calculated by the finite element model. For the measured response, This is the regularization coefficient.

[0109] Furthermore, the graded early warning threshold system described in step 9 includes three levels: observation level, warning level, and emergency level, corresponding to different early warning response measures.

[0110] Observational warning: The probability of damage is within The abnormality may occur within the normal fluctuation range, or the monitored indicator may exceed the normal fluctuation range but not the limit. Response measures include marking the abnormality on the monitoring platform, increasing the sampling frequency, and generating early warning logs.

[0111] Warning level alert: The probability of damage is... If the monitored indicators exceed the design limits, the response measures include pushing early warning information to the management platform, initiating in-depth cloud analysis, and generating a detection recommendation report.

[0112] Emergency warning level: probability of damage This could occur if monitored indicators change drastically and exceed safety limits. Response measures include multi-channel, second-level early warning information transmission, automatic generation of emergency response plans, and establishment of emergency command channels.

[0113] III. Beneficial Effects

[0114] Compared with the prior art, the present invention has the following beneficial effects:

[0115] (1) This invention constructs a three-layer monitoring and diagnosis system architecture of "end-edge-cloud" collaboration, sinking functions such as data preprocessing, feature extraction and preliminary diagnosis to the edge layer, which significantly reduces the amount of original data transmission (experiments show that it can be reduced by more than 85%), reduces network bandwidth pressure, and shortens the diagnosis delay to the millisecond level, realizing real-time monitoring and diagnosis of bridge structure status.

[0116] (2) This invention designs an adaptive edge computing unit based on FPGA, adopting an FPGA+ARM heterogeneous architecture and reconfigurable design to achieve hardware acceleration for signal preprocessing, feature extraction, and lightweight inference. The FPGA uses a systolic array architecture to parallelize convolution operations. Through a 16×16 processing unit (PE) array, 256 multiply-accumulate (MAC) operations can be completed in one clock cycle, meeting the real-time processing requirements under low power conditions. Experiments show that after using INT8 quantization, the MobileNet model inference speed can reach 35 frames / second, with a power consumption of only 3.5W, a model size compression of 4 times, and an inference speed improvement of 3-5 times.

[0117] (3) This invention proposes a multi-scale adaptive data fusion algorithm, which implements fusion at three levels: data level, feature level, and decision level. Data-level fusion employs an improved Kalman filter, which dynamically optimizes the fusion weights based on sensor noise characteristics. Feature-level fusion uses ICEEMDAN decomposition and attention weighting mechanisms, effectively addressing the fusion difficulties caused by the differences in temporal scale, spatial resolution, and physical meaning of multi-source heterogeneous monitoring data. Decision-level fusion uses DS evidence theory, integrating multiple diagnostic results to improve the accuracy of structural state perception. Experiments show that the damage identification accuracy after fusion reaches over 96.5%.

[0118] (4) This invention establishes an edge-cloud collaborative diagnosis mechanism, describing the large-scale multi-access edge computing task offloading problem as a POMDP model, and employing a mean-field multi-agent reinforcement learning algorithm to reduce the dimensionality of the joint action space. Diagnostic tasks are dynamically allocated using damage probability and diagnostic uncertainty as indicators, forming a collaborative diagnosis mode that complements the advantages of real-time edge node diagnosis and deep cloud analysis. Simulation results show that the proposed algorithm outperforms the single-agent task offloading algorithm in terms of task latency and task disconnection rate, and its performance is consistent with the MADDPG algorithm when the dimensionality of the joint action space is reduced.

[0119] (5) This invention constructs a digital twin model based on state-space equations, uses the Guyan reduction method to reduce the model order, and updates the model parameters through real-time data feedback, thereby achieving deep integration of monitoring data and structural mechanism model. Experimental results show that the error rate between the prediction results of the digital twin model and the actual measured displacement data is less than 5%, supporting dynamic mapping of structural state, damage early warning, and maintenance decision-making, and providing a complete solution for the safe operation and maintenance of bridge structures. Attached Figure Description

[0120] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention.

[0121] Figure 2This is a schematic diagram of the "end-edge-cloud" collaborative three-layer architecture of the present invention.

[0122] Figure 3 This is a schematic diagram of the edge computing node hardware architecture of the present invention.

[0123] Figure 4 This is a schematic diagram of the FPGA pulse array accelerator structure of the present invention.

[0124] Figure 5 This is a flowchart of the multi-scale data adaptive fusion algorithm of the present invention.

[0125] Figure 6 This is a schematic diagram of the ICEEMDAN signal decomposition of the present invention.

[0126] Figure 7 This is a schematic diagram of the edge-cloud collaborative diagnostic mechanism of the present invention.

[0127] Figure 8 This is a schematic diagram of the digital twin model based on state-space equations of the present invention.

[0128] Figure 9 This is a flowchart of the hierarchical early warning and response mechanism of the present invention. Detailed Implementation

[0129] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0130] Example 1

[0131] This embodiment provides a method for collaborative monitoring and real-time diagnosis of bridge structures based on edge computing, which is applied to the structural health monitoring system of a long-span railway bridge.

[0132] Step 1: Construct a collaborative three-tier monitoring and diagnostic system architecture for "device-edge-cloud".

[0133] like Figure 2 As shown, a three-layer architecture consisting of a perception layer, an edge layer, and a cloud platform layer is constructed.

[0134] The perception layer consists of a collaborative sensing network composed of various intelligent sensors deployed at key parts of the bridge structure. In this embodiment, 32 intelligent sensor nodes are deployed at key sections such as the mid-span, supports, and pier tops of the bridge. These include: 16 fiber optic strain sensors (for stress and strain monitoring), 8 MEMS accelerometers (for vibration monitoring), 2 BeiDou / GNSS receivers (for overall displacement monitoring), 4 temperature and humidity sensors (for environmental monitoring), and 2 high-definition industrial cameras (for surface defect monitoring). All sensor nodes have an adaptive sampling frequency adjustment function, which can dynamically adjust the data acquisition frequency according to the instructions of the edge computing nodes.

[0135] The edge layer consists of four edge computing nodes deployed on the bridge site, each connected to eight smart sensors. For example... Figure 3 As shown, the edge computing node adopts an FPGA+ARM heterogeneous computing architecture, including: an FPGA programmable logic unit (Xilinx Zynq Ultra Scale + MPSoC), an ARM Cortex-A53 quad-core processor, 4GB DDR4 memory, 32GB eMMC storage, a gigabit Ethernet interface, and a 5G communication module. The FPGA is responsible for parallel data processing tasks, including adaptive filtering, FFT transformation, matrix operations, and lightweight neural network inference; the ARM is responsible for task scheduling, communication management, and protocol conversion.

[0136] like Figure 4 As shown, a Systolic Array accelerator is implemented in the FPGA, employing a 16×16 Processing Unit (PE) array. Each PE contains a multiplier and an accumulator, capable of performing 256 multiply-accumulate (MAC) operations in one clock cycle. A three-level memory architecture is used: on-chip BRAM stores weights and activation values, off-chip DDR stores feature maps, and through tiled convolution technology, the 64×64 feature map is divided into 8×8 sub-blocks, increasing the on-chip data reuse rate to over 90%.

[0137] The cloud platform layer is a remote computing platform deployed in the cloud, connecting four edge computing nodes and providing functions such as data aggregation and storage, digital twin modeling, complex diagnostic analysis, model training and updating, and decision support.

[0138] Step 2: Multi-source monitoring data acquisition

[0139] The intelligent sensors in the perception layer collect multi-source monitoring data of the bridge structure according to a preset sampling frequency, and transmit the collected data to the corresponding edge computing nodes via ZigBee or LoRa wireless communication. In this embodiment, the fiber optic strain sensor has a sampling frequency of 100Hz, the MEMS accelerometer has a sampling frequency of 500Hz, the Beidou / GNSS receiver has a sampling frequency of 10Hz, the temperature and humidity sensor has a sampling frequency of 0.1Hz, and the high-definition industrial camera acquires one image every 10 minutes under normal circumstances, triggering continuous acquisition when an abnormal event is detected.

[0140] Step 3: Data Preprocessing

[0141] Edge computing nodes preprocess the received multi-source monitoring data, including:

[0142] (1) Data cleaning: using The principle is to identify and remove abnormal data points caused by sensor malfunctions or communication errors, i.e., when a data point meets the following conditions... If it is found to be abnormal, it will be removed.

[0143] (2) Outlier removal: Smoothing the data sequence after outlier removal.

[0144] (3) Missing value compensation: Linear interpolation is used to compensate for transiently missing data; for data with more than 10 consecutive missing sampling points, Gaussian process regression model is used for prediction compensation. Gaussian process regression uses a squared exponential kernel. The hyperparameters are obtained by maximizing the marginal likelihood estimation.

[0145] (4) Time synchronization: Based on the IEEE 1588 Precision Time Protocol (PTP), all sensor data are synchronized to a unified time reference, with synchronization accuracy reaching the microsecond level.

[0146] Step 4: Adaptive Fusion of Multi-Scale Data

[0147] like Figure 5 As shown, the edge computing nodes perform multi-scale adaptive data fusion on the preprocessed multi-source monitoring data, including three levels: data-level fusion, feature-level fusion, and decision-level fusion.

[0148] (1) Data-level fusion: Fusion of multi-sensor measurements of the same physical quantity. In this embodiment, the measurements from fiber optic strain sensors and nanoscale strain gauges are fused using an improved Kalman filter algorithm. State vector For the true strain value, the observation vector These are the measurements from two sensors. The filtering algorithm employs an adaptive covariance matching method, estimating the observation noise covariance matrix in real time based on the innovation sequence. The fusion weights are dynamically adjusted.

[0149] (2) Feature-level fusion: includes the following sub-steps:

[0150] Step 4.2.1: Use interpolation to synchronize data from different sampling frequencies to a unified time base. Time window length. ,in The fundamental frequency of the structure is taken (the fundamental frequency of this bridge is approximately 0.8Hz). Take 3, and calculate as follows Seconds, rounded to 4 seconds.

[0151] Step 4.2.2: Extract physical features and statistical features from each type of data.

[0152] For vibration data, such as Figure 6As shown, the ICEEMDAN algorithm is used for decomposition to obtain several IMF components. The Hurst index is calculated for each IMF component, and then filtered. The components are reconstructed. The inherent frequencies are extracted from the reconstructed signal. Damping ratio , vibration shape Modal parameters. Modal parameter identification employs frequency domain decomposition to analyze the vibration transfer function. Singular value decomposition is performed, and the frequency corresponding to the peak value is the natural frequency.

[0153] For strain data, extract the stress level. Stress amplitude Stress cycle counting Fatigue-related characteristics were assessed. Stress cycle counting was performed using the rainflow counting method, counting the number of cycles at each stress amplitude for fatigue life assessment.

[0154] For displacement data, extract the maximum deflection. Residual deformation Deformation characteristics such as linear change rate.

[0155] For temperature data, extract the temperature gradient. Average temperature Features such as...

[0156] For image data, a deep learning-based semantic segmentation model is used to identify cracks and extract crack lengths. ,width ,area Defect characteristics such as direction and orientation are considered. Crack width is calculated using pixel-level measurement, which is then converted into physical dimensions based on camera calibration parameters.

[0157] Step 4.2.3: A deep neural network is used to fuse the extracted multi-source features. A multi-input neural network architecture is designed, with each type of feature input into an independent sub-network (a 2-layer fully connected network with 128 hidden layers) for feature transformation. An attention-weighted mechanism is used in the fusion layer, with attention weights... The computation is performed using a learnable attention network. The final output is a 128-dimensional multimodal fusion feature vector.

[0158] (3) Decision-level fusion: When combining diagnostic results from edge nodes and the cloud, the DS evidence theory is used for fusion. Identification framework ={normal, minor damage, moderate damage, severe damage}, construct a BPA function based on the diagnostic results of edge nodes and the cloud, obtain the fused probability allocation through Dempster synthesis rules, and take the state category corresponding to the highest probability as the final decision.

[0159] Step 5: Real-time diagnosis of edge nodes

[0160] Edge computing nodes perform real-time diagnostics on structural state feature vectors based on lightweight diagnostic models. In this embodiment, three lightweight diagnostic models are deployed:

[0161] (1) Anomaly Detection Model: Based on an autoencoder, the encoder compresses 128-dimensional features to 32-dimensional features, and the decoder reconstructs the original features. The reconstruction error is used as the anomaly score. Anomaly is determined when the reconstruction error exceeds a threshold. Anomaly Score The calculation is as follows:

[0162]

[0163] (2) Damage classification model: Based on a CNN-BiLSTM hybrid architecture, it is used to identify damage types (cracks, loosening, corrosion, settlement, etc.). The model structure includes: 2 convolutional layers (3×3 kernel size, 32 / 64 channels), 2 BiLSTM layers (128 hidden units), and 1 fully connected output layer (Softmax activation), which outputs the probability of each category. .

[0164] (3) Damage assessment model: Based on a lightweight convolutional neural network, it is used to quantify the degree of damage (slight, moderate, severe) and output the damage level and confidence level.

[0165] All models underwent 8-bit fixed-point quantization (INT8), and the optimal scaling factor was determined using the KL divergence calibration method during the quantization process. and zero offset After quantization, the model size is reduced by 4 times, and the inference speed is increased by more than 3 times.

[0166] The edge node performs a diagnosis on the data for each time window (4 seconds) and outputs preliminary diagnostic results, including structural state category and damage probability. And diagnostic uncertainty estimation Diagnostic uncertainty is estimated using the Monte Carlo dropout method: dropout is enabled during inference. The second forward propagation calculates the prediction variance as uncertainty:

[0167]

[0168] Step 6: Edge-Cloud Collaborative Diagnosis

[0169] like Figure 7 As shown, the edge computing node determines whether the diagnostic task needs to be unloaded to the cloud platform based on the preliminary diagnostic results and the dynamic task unloading strategy.

[0170] This embodiment models the task offloading problem as a POMDP. In time slots Edge nodes Observations Includes: current damage probability Diagnostic uncertainty Node resource status (CPU utilization, memory utilization, battery level). Actions For the unloading decision (0: upload only the result, 1: upload the feature vector, 2: upload the original data). The reward function is defined as:

[0171]

[0172] The decision network is trained using a mean-field multi-agent reinforcement learning algorithm, with threshold parameters set. , , The decision-making rules are as follows:

[0173] when and When the condition is normal, the edge computing node independently completes the diagnosis, only uploading the structural state feature vector and the diagnosis conclusion to the cloud platform.

[0174] when or When an anomaly is detected, it indicates a possible but uncertain situation, triggering cloud-based assisted diagnosis. Edge computing nodes package and upload the raw monitoring data (raw time series and images), structural state feature vectors, and preliminary diagnostic results to the cloud platform.

[0175] when When this occurs, it indicates that an anomaly has been clearly detected, triggering in-depth diagnostics in the cloud. The edge computing node uploads detailed data, and the cloud platform initiates the in-depth analysis process.

[0176] Step 7: Cloud-based in-depth diagnostics

[0177] The cloud platform receives data uploaded by edge computing nodes and performs in-depth analysis based on a complex diagnostic model. In this embodiment, the following models are deployed in the cloud:

[0178] (1) Crack recognition model based on ResNet-50: Input high-resolution image, output crack segmentation map and crack parameters. The model is pre-trained on a large bridge crack dataset and fine-tuned using transfer learning.

[0179] (2) Damage recognition model based on ICEEMDAN-CNN: The original vibration data is input, first decomposed using ICEEMDAN to obtain IMF components, then filtered and input into a 1D-CNN network, outputting the damage location and type. The model loss function is... ,in .

[0180] (3) Parameter inversion model based on finite element model correction: Combine the measured response and the finite element model, and use the Bayesian model update method to invert the structural parameters (elastic modulus, moment of inertia of section, boundary condition stiffness, etc.).

[0181] The cloud-based deep diagnostic output includes: damage location (marked on the digital twin model), damage type, damage extent (quantified value), and structural safety margin.

[0182] Step 8: Visualizing the Digital Twin

[0183] like Figure 8 As shown, the cloud platform, based on a digital twin model, maps the deep diagnostic results to the digital twin model of the bridge structure, thereby enabling a visual display of the structural status.

[0184] In this embodiment, the digital twin model constructs a three-dimensional geometric model of the bridge based on BIM technology (Revit modeling), and integrates a finite element model based on state-space equations on the geometric model. The finite element model contains 1256 nodes and 2348 elements, and the degrees of freedom are reduced to 156 master degrees of freedom (corresponding to sensor positions) using the Guyan reduction method.

[0185] Structural dynamics equations Transformed into state-space equation form:

[0186]

[0187]

[0188] State vector The dimension is 312.

[0189] Real-time monitoring data is mapped to corresponding locations in the digital twin model, and model parameters are corrected through model update algorithms. (Mainly the elastic modulus correction coefficient), to make the model response approximate the measured response:

[0190]

[0191] The updated digital twin model calculates structural stress distribution, deformation state, and safety margin in real time, displaying the data as a color temperature map in a 3D visualization interface. Users can intuitively view real-time monitoring values, historical variation curves, and diagnostic results for each measuring point within the 3D model.

[0192] Step 9: Tiered Early Warning and Response

[0193] like Figure 9As shown, the cloud platform uses a decision-level fusion method (DS evidence theory) to fuse the preliminary and in-depth diagnostic results to obtain the final diagnostic conclusion, and triggers the corresponding early warning response according to the preset hierarchical early warning threshold system.

[0194] The tiered early warning thresholds are set as follows:

[0195] Observational warning: probability of damage Or the monitored indicators exceed the normal fluctuation range but do not exceed the limit.

[0196] Warning level alert: Probability of damage Or the monitoring indicators exceed the design allowable values.

[0197] Emergency warning level: probability of damage Or, the monitoring indicators change drastically and exceed the safety limits.

[0198] Response measures:

[0199] Observation-level response: Mark abnormal measurement points and time periods on the monitoring platform to remind management personnel to pay attention; increase the sampling frequency in the relevant area (from 100Hz to 200Hz); generate early warning logs for future reference.

[0200] Warning-level response: Pushes early warning information (including anomaly type, location, and severity) to the management platform and relevant personnel; initiates cloud-based in-depth analysis process; invokes refined diagnostic models; generates a testing recommendation report, recommending key areas and methods for professional testing.

[0201] Emergency Response: Enables second-level early warning information transmission through multiple channels such as large screens, mobile terminals, and emergency broadcasts; automatically generates emergency response plans (including traffic control recommendations, emergency reinforcement measures, and on-site testing plans); and establishes an emergency command channel to achieve multi-party collaborative action.

[0202] Step 10: Model Collaborative Update

[0203] The cloud platform continuously trains and optimizes the diagnostic model using aggregated multi-bridge data. In this embodiment, the cloud model is incrementally trained every quarter using newly collected data to improve model performance.

[0204] For lightweight models deployed on edge nodes, a knowledge distillation technique is employed. The large model in the cloud serves as the teacher network, training smaller edge models to mimic the output of the teacher network. The distillation loss function is:

[0205]

[0206] in, For cross-entropy loss, Let KL divergence be the KL divergence. This is the balance coefficient (taken as 0.3).

[0207] The model update package is delivered to edge nodes via over-the-air (OTA) technology, and hot updates of the model are completed during the low-load period at night, enabling continuous evolution of diagnostic capabilities.

[0208] Example 2

[0209] This embodiment provides a method for collaborative monitoring and real-time diagnosis of bridge structures based on edge computing, which is applied to the intensive monitoring of highway bridge groups.

[0210] Compared with Example 1, the difference in this example is:

[0211] (1) Perception layer: A total of 120 smart sensor nodes are deployed on 5 highway bridges, with 24 nodes deployed on each bridge. In addition to the sensor types listed in Example 1, a vehicle dynamic weighing sensor (WIM) is added to monitor vehicle load.

[0212] (2) Edge layer: Two edge computing nodes are deployed on each bridge, for a total of 10 nodes. The edge nodes are interconnected through a Mesh network to form a regional edge computing network, which supports cross-bridge collaborative diagnosis.

[0213] (3) Cloud platform layer: Deploy a regional cloud platform to aggregate data from 10 edge nodes and realize centralized monitoring and unified management of 5 bridges.

[0214] (4) Collaborative diagnosis: Add cross-bridge collaborative diagnosis function. When an anomaly is detected on a certain bridge, the edge computing node sends a collaborative diagnosis request to the edge nodes of other bridges to compare and analyze the response of similar structures on different bridges to assist in anomaly identification.

[0215] (5) Digital twin: Construct a regional digital twin platform containing 5 bridges to support lateral comparative analysis between bridges and regional structural safety assessment.

[0216] The remaining steps are the same as in Example 1.

[0217] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for collaborative monitoring and real-time diagnosis of bridge structures based on edge computing multi-source information, characterized in that, Includes the following steps: Step 1: Construct a three-layer monitoring and diagnostic system architecture that is collaborative between the "device, edge, and cloud", including a perception layer, an edge layer, and a cloud platform layer; the perception layer consists of a collaborative perception network composed of various intelligent sensors deployed in key parts of the bridge structure. The edge layer consists of edge computing nodes deployed at the bridge site, with each edge computing node connected to multiple smart sensors; the cloud platform layer is a remote cloud computing platform that connects to multiple edge computing nodes. Step 2: The intelligent sensors in the perception layer collect multi-source monitoring data of the bridge structure and send the collected data to the corresponding edge computing nodes; Step 3: The edge computing node preprocesses the received multi-source monitoring data, including data cleaning, outlier removal, missing value compensation, and time synchronization, to obtain preprocessed multi-source monitoring data; Step 4: Edge computing nodes perform multi-scale adaptive data fusion on the preprocessed multi-source monitoring data to obtain structural state feature vectors; the multi-scale adaptive data fusion includes three levels: data-level fusion, feature-level fusion, and decision-level fusion. Step 5: Edge computing nodes perform real-time diagnosis of the structural state feature vector based on a lightweight diagnostic model, and output preliminary diagnostic results; the preliminary diagnostic results include structural state category and damage probability. And diagnostic uncertainty estimation ; Step 6: Based on the preliminary diagnosis results and the dynamic task offloading strategy, the edge computing node determines whether the diagnosis task needs to be offloaded to the cloud platform. If so, the original monitoring data, structural state feature vector, and preliminary diagnosis results are uploaded to the cloud platform to trigger in-depth cloud diagnosis. Otherwise, only the preliminary diagnosis results are uploaded to the cloud platform. Step 7: The cloud platform receives the data uploaded by the edge computing nodes, performs in-depth analysis based on the complex diagnostic model, and outputs in-depth diagnostic results; Step 8: The cloud platform maps the deep diagnostic results to the digital twin model of the bridge structure based on the digital twin model, so as to realize the visualization of the structural status. Step 9: Based on the preliminary diagnosis results and the in-depth diagnosis results, the cloud platform uses a decision-level fusion method to fuse the results, obtain the final diagnosis conclusion, and trigger the corresponding early warning response according to the preset hierarchical early warning threshold system; Step 10: The cloud platform uses the aggregated multi-bridge data to continuously train and optimize the diagnostic model, and regularly distributes the updated lightweight model to edge computing nodes to achieve continuous evolution of diagnostic capabilities.

2. The method according to claim 1, characterized in that, The edge computing node described in step 1 adopts an FPGA+ARM heterogeneous computing architecture, where the FPGA is responsible for parallel data processing tasks, including signal filtering, feature extraction, and lightweight neural network inference; the ARM is responsible for task scheduling, communication management, and protocol conversion; the edge computing node supports dynamic partial reconfiguration of FPGA logic functions and can dynamically load different hardware acceleration modules according to the monitoring task requirements.

3. The method according to claim 1, characterized in that, The data cleaning described in step 3 uses The principle is to identify outliers when data points meet the following conditions. When an error is detected, it is identified as abnormal and removed. For the current data point, The mean within the sliding window. The standard deviation is the value within the sliding window; the missing value compensation uses a Gaussian process regression model, and its covariance function uses a squared exponential kernel. .

4. The method according to claim 1, characterized in that, The data-level fusion described in step 4 employs an improved Kalman filter algorithm, whose state equation is: The observation equation is ; An adaptive covariance matching method is used to dynamically optimize the fusion weights and estimate the observation noise covariance matrix in real time. ,in For the innovation vector, It is a forgetting factor.

5. The method according to claim 1, characterized in that, The feature-level fusion in step 4 includes: using the ICEEMDAN algorithm to decompose the vibration data to obtain the intrinsic mode function components. Calculate the Hurst exponent Sensitive components are selected; attention-weighted fusion is performed using a multi-input neural network. ,in For attention weights.

6. The method according to claim 1, characterized in that, The decision-level fusion described in step 4 adopts the DS evidence theory, and the identification framework is set. , No. The basic probability distribution of each piece of evidence is: ,satisfy Dempster's synthesis rules were adopted. Integrating multiple pieces of evidence, including conflicting factors .

7. The method according to claim 1, characterized in that, The lightweight diagnostic model described in step 5 undergoes 8-bit fixed-point quantization. The quantization process is as follows: ,in For floating-point values, The quantized integer value. Scaling factor Zero-point offset; diagnosing uncertainty Estimated using the Monte Carlo dropout method: .

8. The method according to claim 1, characterized in that, The dynamic task offloading strategy described in step 6 describes the task offloading problem as a partially observable Markov decision process, and uses a mean-field multi-agent reinforcement learning algorithm to approximate the Q-value function as follows: ,in The action is approximated by the mean field; with damage probability and diagnostic uncertainty Task allocation is dynamically determined based on the indicators.

9. The method according to claim 1, characterized in that, The digital twin model described in step 8 is constructed based on state-space equations and structural dynamics equations. Transform into , , where the state vector The Guyan reduction method is used to reduce the model order, and the reduced stiffness matrix is: Correcting model parameters through model update algorithms .

10. The method according to claim 1, characterized in that, The model collaborative update described in step 10 employs knowledge distillation technology, with the distillation loss function being: ,in For cross-entropy loss, Let KL divergence be the KL divergence. The model update package is distributed to edge computing nodes via over-the-air (OTA) download technology, serving as a balancing factor.

Images