Bridge early warning method and system based on physical information neural network and machine vision

Abstract

The invention proposes a bridge early warning method and system based on a physical information neural network and machine vision, and relates to the technical field of bridge early warning, and the method comprises the steps: setting a visual data collection system on a bridge; designing a known load calibration test, and collecting a preliminary calibration data set; collecting a preliminary random load data set; using an error correction algorithm to obtain a corrected random load data set; establishing a causal association database covering the dynamic response relationship between the vehicle and the bridge; constructing a physical information neural network model, and training a physical information neural network based on the causal association database to obtain a trained physical information neural network model; and acquiring bridge dynamic response data in real time, inputting the bridge dynamic response data into the trained physical information neural network model, performing online evaluation on the bridge operation state, checking abnormal response and generating an early warning signal. According to the method, the physical law and the machine learning algorithm are combined, and accurate online evaluation and anomaly detection of the bridge operation state are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge early warning, and in particular to a bridge early warning method and system based on physical information neural network and machine vision. Background Art

[0002] As a key hub in transportation construction, bridges are not only important infrastructure for national economic and social development, but also a comprehensive reflection of a country or region's economic strength, scientific and technological level, and productivity development. In the development of today's economy and society, bridges play a vital role. As a major engineering structure, the service life of a bridge is usually several decades to a hundred years. However, the coupling of factors such as environmental erosion, material aging, long-term load effects, fatigue effects, and mutation effects will inevitably lead to the accumulation of damage and resistance attenuation of bridge structures, thereby reducing their ability to resist natural disasters and even normal loads, and in extreme cases even causing catastrophic accidents.

[0003] These major safety accidents have caused huge losses to the country's economic property and people's lives. In order to ensure the operational safety of bridges, bridge management units usually use manual inspection techniques. However, the data obtained by manual inspection lacks continuity and real-time, and often cannot detect defects in time, especially some hidden defects are difficult to detect, making it difficult to effectively ensure the safety of bridges. A survey by the Federal Highway Commission of the United States showed that "56% of the evaluation results obtained by manual visual inspection are inaccurate." For example, although the Mississippi River Bridge in the United States was inspected in 2005 and 2006 and no structural safety hazards were found at the time, it still collapsed in 2007. This series of serious bridge safety accidents highlights the shortcomings of manual detection technology, and also highlights the necessity and importance of dynamic bridge early warning systems.

[0004] At present, bridge dynamic early warning systems mainly include the following types, but their application is limited by their respective shortcomings: Methods based on threshold judgment: Traditional bridge health monitoring systems usually rely on fixed threshold judgment methods. These methods judge the monitoring data by pre-setting safety thresholds, such as deflection, stress or vibration amplitude. Once the detection value exceeds the threshold range, the system will trigger an alarm. However, this method has obvious defects: Lack of accuracy: The state of the bridge is usually affected by multiple factors such as load, ambient temperature, humidity, etc. A single threshold is difficult to fully reflect the actual health status of the bridge, which may lead to false alarms or omissions. Unable to respond dynamically: The threshold setting is usually static, while the operating state of the bridge is highly dynamic. For example, the dynamic response caused by an instantaneous overloaded vehicle cannot be captured in time, and important abnormal information is easy to miss.

[0005] Finite element model modification method: The finite element model (FEM) is a commonly used theoretical tool in bridge structure analysis. By establishing a physical model of the bridge, its response under different loads is simulated. However, the finite element model has the following problems in practical applications: Time-consuming: The establishment and modification of the model requires a lot of manual participation and experimental data support. Especially when facing complex bridge structures or multiple load conditions, the calculation process is very time-consuming and it is difficult to meet real-time requirements. Difficult to integrate multiple factors: Finite element models are usually only for specific mechanical properties (such as statics or dynamics), while the operation status of the bridge involves complex multi-physical field coupling, such as vehicle loads, wind loads, temperature changes, etc. These factors are often simplified or ignored in traditional models, resulting in insufficient accuracy and reliability of the analysis results.

[0006] Data-driven methods: In recent years, data-driven bridge monitoring methods have gradually emerged. These methods analyze large amounts of sensor data or monitor historical data and use machine learning or statistical methods to evaluate the state of the bridge. Although these methods have certain flexibility, they still have the following limitations: Unable to integrate physical meaning: Purely data-driven models only focus on the correlation between data and ignore the clear physical laws that exist in bridge structures, such as load-deflection relationships or vibration modal characteristics. The prediction results of this method lack physical basis and may fail in practical applications. Ignoring causal relationships: Data-driven methods are mostly based on empirical statistics, focusing on the prediction of results, and failing to deeply analyze the causal relationship between input loads and bridge responses, making it difficult to support root cause diagnosis of bridge anomalies.

[0007] Deep learning method: Due to its powerful nonlinear fitting ability, deep learning technology has been applied to the field of bridge health monitoring in recent years, attempting to analyze and predict complex data through neural networks. However, this method also has obvious defects: the "black box" problem of the model: the neural network structure is complex, and the internal operation mechanism of the model is difficult for users to understand, resulting in poor interpretability of the results, reducing users' trust in the prediction results, especially in bridge monitoring scenarios with extremely high safety requirements. It is difficult to reflect physical laws: The training of deep learning models mainly relies on data, and lacks physical constraints on bridge structures and mechanical properties. As a result, although the model performs well in data fitting, it is prone to deviations or even errors when generalized to unseen working conditions.

[0008] In summary, the traditional bridge health monitoring methods have obvious deficiencies in accuracy, real-time performance, multi-factor analysis, and integration of physical laws, which limits their application in dynamic assessment of complex bridges. Therefore, a new method that can combine physical laws with efficient algorithms is urgently needed to make up for the shortcomings of existing methods and improve the reliability and real-time performance of bridge early warning systems. Summary of the invention

[0009] In view of this, the present invention proposes a bridge early warning method and system based on physical information neural network and machine vision. By real-time collection of bridge dynamic response data and vehicle trajectory information, combined with physical laws and machine learning algorithms, accurate online evaluation and anomaly detection of the bridge operation status can be achieved.

[0010] The technical solution of the present invention is achieved in this way: On the one hand, the present invention provides a bridge early warning method based on physical information neural network and machine vision, comprising: S1. Setting up a visual data acquisition system on the bridge, including bridge dynamic displacement acquisition equipment and trajectory visual acquisition equipment; S2. Design a known load calibration test, including selecting a standardized specific load vehicle and designing a predetermined trajectory. The visual data acquisition system collects dynamic response data under known load conditions to form a preliminary calibration data set; S3, collecting vehicle trajectories, vehicle types and corresponding bridge response data under natural traffic flow to form a preliminary random load data set; S4, conduct a known load calibration test again, collect the dynamic response of the bridge by the visual data acquisition system, form a second calibration data set, compare the preliminary calibration data set with the second calibration data set, identify systematic errors and random errors, and use the error correction algorithm to correct the preliminary random load data set to obtain a corrected random load data set; S5. Establish a causal correlation database covering the dynamic response relationship between the vehicle and the bridge through the corrected random load data set, the preliminary calibration data set and the second calibration data set; S6. Construct a physical information neural network model, train the physical information neural network using a supervised learning method based on a causal association database, optimize network parameters, and obtain a trained physical information neural network model; S7. Collect bridge dynamic response data in real time and input it into the trained physical information neural network model to conduct online evaluation of the bridge operation status, check abnormal responses and generate early warning signals.

[0011] On the basis of the above technical solution, preferably, the bridge dynamic displacement acquisition equipment includes a digital image correlation acquisition device deployed under the bridge, which has a multi-scale image acquisition unit and a real-time optical flow algorithm module for non-contact monitoring of the deflection changes of key parts of the bridge under different frequency loads; the trajectory visual acquisition equipment includes a high-resolution camera array and a multi-view synchronous collector deployed on the bridge deck and the bridge head, and the camera array obtains the vehicle shape, speed and driving trajectory information through the Yolo v8 target detection model, and fuses these visual data with the data output by the bridge dynamic displacement acquisition equipment on the same time basis.

[0012] On the basis of the above technical solution, preferably, the error correction algorithm in step S4 is as follows: Step 1: Define the initial error vector ,in is the initial system error, is the initial random error, is the initial environmental error; Step 2: Calculate the intermediate correction function at the kth iteration correction ,in, For the preliminary calibration dataset, is the second calibration data set, is the adaptive convergence factor, Based on multidimensional nonlinear mapping, the previous iteration result and , After coupling operation, a new correction vector is output ; Step 3: Substitute the intermediate correction result into the correction formula: , In the formula, is the exponential penalty factor, represents the vector norm, It is a quadratic coupling correction function, which performs nonlinear compensation according to the distribution characteristics of vehicle load effects and environmental disturbance characteristics; Step 4: After several iterations, when or Output the final correction result when it is lower than the predetermined threshold , and Applied to the preliminary random load data set, a revised random load data set is obtained.

[0013] Based on the above technical solution, preferably, step S5 includes: S51, integrating the corrected random load data set, the preliminary calibration data set and the second calibration data set, associating the vehicle load parameters with the bridge response parameters according to the timestamps, and obtaining a synchronous record of multi-source data; S52, performing multi-level feature extraction on the above synchronous records to obtain a data layer, a feature layer and an association layer respectively; wherein the data layer includes the original data of the vehicle node, the response node and the environment node, the feature layer includes frequency domain and time domain features, and the association layer is used to record the direct or indirect causal relationship between the nodes; S53. Extract the main causal chain based on the causal strength to form the direct and indirect causal relationship between vehicle load and bridge response; use optimization algorithm to optimize the accuracy and completeness of the causal chain; S54. Enter the constructed causal chain and association relationship into the graph database to form a causal association database covering a multi-level data structure.

[0014] Based on the above technical solution, preferably, the causal strength calculation formula is: , In the formula, represents the causal strength from node i to node j; is the mutual information between node i and node j; is the directional consistency from node i to node j; A nonlinear parameter to adjust the causal strength; The Bayesian optimization algorithm is used to adjust the weight of the extracted causal chain, and the formula is defined as: , in, is the final causal weight from node i to node j; m is the total number of nodes that have a causal relationship with node i.

[0015] On the basis of the above technical solution, preferably, the physical information neural network model is a multi-branch hybrid structure, including: Physical branch: It includes an input layer, two fully connected hidden layers, and an output layer. Each hidden layer applies a physical constraint activation function. The physical branch is used to process the embedded bridge mechanics equations and output the physical constraint results. Data branch: includes input layer, multi-scale convolution layer, attention mechanism layer and fully connected hidden layer; multi-scale convolution layer is used to extract multi-scale features; attention mechanism layer is used to autonomously learn the importance of different features; data branch is used to process high-frequency visual data and extract deep feature information; Fusion layer: contains an adaptive weight fusion unit, which dynamically weights and fuses the outputs of the physical branch and the data branch. The fusion layer learns weight parameters to achieve effective information integration. Output layer: integrates the fusion results of the physical branch and the data branch, and outputs the bridge operation status assessment.

[0016] On the basis of the above technical solution, preferably, the bridge mechanics equation in the physical branch includes static constraints and dynamic constraints, wherein: The static constraint formula is: , In the formula, x is the spatial coordinate position, that is, the position parameter of each measuring point in the bridge; represents the mass matrix of the bridge; represents the damping matrix; represents the stiffness matrix; represents the displacement vector of the bridge; is the external load vector; express Second-order partial derivative with respect to time; express First-order partial derivative with respect to time; The dynamic constraint formula is: , In the formula, represents the natural frequency of the bridge; represents the vibration mode vector of the bridge; is the stiffness matrix.

[0017] Based on the above technical solution, preferably, the loss function used in the training of the physical information neural network model is: , , , , In the formula, is the total loss function; For data driven losses; Loss of physical restraint; is the regularization loss; , , is the adaptive weight coefficient; represents the sth sample; is the number of samples; represents the actual value of the sample, that is, the real response data of the bridge at the sth position; Represents the model prediction value, that is, the bridge response data predicted by the model; To adjust the parameters; , , , , is the physical parameter corresponding to sample s; is the standardized parameter; is the regularization coefficient; For the model Layer weight parameter; Q is the number of network layers.

[0018] Based on the above technical solution, preferably, step S7 includes: S71, acquiring dynamic response data generated by the bridge during operation in real time through a visual data acquisition system, and performing time synchronization and preliminary preprocessing on the acquired dynamic response data; S72, converting the preprocessed dynamic response data into the input format required by the physical information neural network model, and inputting the data into the trained physical information neural network model, calculating the current operating state parameters of the bridge through the forward propagation process, and the model outputs the health assessment results of the bridge according to the embedded physical constraints and causal association database, including the current deflection, vibration mode and potential structural abnormality indicators; S73. Based on the health assessment results output by the model, the preset safety standards and thresholds are compared to determine whether the bridge operation status is normal, and a multi-level assessment mechanism is used to distinguish between slight abnormalities, obvious abnormalities and serious abnormalities; S74. Generate warning signals of corresponding levels based on the abnormality assessment results, including level 1 warning, level 2 warning and level 3 warning, which correspond to slight abnormalities, obvious abnormalities and severe abnormalities respectively. Each level of warning signal contains specific abnormal parameter values, abnormality categories and recommended response measures. The warning signal is transmitted in real time to the bridge management and operation and maintenance system through wireless communication or wired network.

[0019] On the other hand, the present invention also provides a bridge early warning system based on physical information neural network and machine vision, the system is used to execute any of the above methods, the system comprises: The data acquisition module integrates bridge dynamic displacement acquisition equipment and trajectory visual acquisition equipment to collect dynamic response data of the bridge and vehicle trajectory data in real time; The dual calibration module performs at least two known load calibration tests and natural traffic flow collection, evaluates the difference of random load data and calls the error correction algorithm for dynamic correction; The causal relationship database module stores, builds causal relationships, and iteratively updates multi-source heterogeneous data based on a graph database structure containing vehicle nodes, bridge response nodes, and environment nodes; Physical information neural network module, including a physical information neural network model with multi-branch hybrid structure, to evaluate the bridge operation status; The real-time evaluation and early warning module inputs dynamic response data into the trained neural network model in real time; generates multi-level early warning signals according to the model output, and transmits and feedbacks the signals through the early warning setting unit; Communication and storage module, used to transmit warning signals to the bridge management and operation and maintenance system; The user interface and monitoring module provides a visual bridge operation status monitoring interface; displays real-time data, warning information and historical records.

[0020] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention combines physical information neural network with machine vision technology to construct a bridge early warning method with multi-source data fusion. The method uses a dual calibration mechanism and an error correction algorithm to improve data quality, stores multi-level data structures through a causal association database, and uses a physical information neural network model with a multi-branch hybrid structure for state assessment. It achieves accurate monitoring of the bridge operation status and timely early warning, effectively overcoming the shortcomings of traditional early warning methods in terms of accuracy, real-time performance, and physical interpretability. (2) The present invention adopts a visual data acquisition system consisting of a digital image correlation acquisition device and a high-resolution camera array, combined with the Yolo v8 target detection model, to achieve non-contact real-time monitoring of bridge dynamic displacement and vehicle trajectory, avoiding the problem of difficult installation and maintenance of traditional contact sensors, while improving the accuracy and reliability of data acquisition; (3) The error correction algorithm designed by the present invention defines the three-dimensional error vector of system error, random error and environmental error, and uses multi-dimensional nonlinear mapping and quadratic coupling correction function for iterative optimization, which effectively eliminates various errors in the data acquisition process and significantly improves the accuracy of monitoring data; (4) The causal association database constructed by the present invention adopts a graph database structure, calculates the causal strength through mutual information and directional consistency, and uses Bayesian optimization to adjust the weight, thereby achieving accurate expression and efficient storage of the complex causal relationship between vehicle loads and bridge responses; (5) The physical information neural network model of the present invention adopts a multi-branch hybrid structure of physical branches and data branches. By embedding the bridge mechanics equation as a physical constraint and combining multi-scale convolution and attention mechanisms to process visual data, it realizes the effective fusion of physical laws and data features, thereby improving the prediction accuracy and generalization ability of the model. (6) The present invention adopts a multi-level evaluation mechanism and a graded warning strategy, which can accurately distinguish between minor abnormalities, obvious abnormalities and serious abnormalities, and generate warning signals of corresponding levels, including specific abnormal parameter values, categories and recommended measures, providing a scientific decision-making basis for bridge management departments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 is a flow chart of the method of the present invention; Figure 2 It is a model structure diagram of the present invention; Figure 3 It is a system framework diagram of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] like Figure 1 As shown, the present invention provides a bridge early warning method based on physical information neural network and machine vision, comprising: S1. Setting up a visual data acquisition system on the bridge, including bridge dynamic displacement acquisition equipment and trajectory visual acquisition equipment; S2. Design a known load calibration test, including selecting a standardized specific load vehicle and designing a predetermined trajectory. The visual data acquisition system collects dynamic response data under known load conditions to form a preliminary calibration data set; S3, collecting vehicle trajectories, vehicle types and corresponding bridge response data under natural traffic flow to form a preliminary random load data set; S4, conduct a known load calibration test again, collect the dynamic response of the bridge by the visual data acquisition system, form a second calibration data set, compare the preliminary calibration data set with the second calibration data set, identify systematic errors and random errors, and use the error correction algorithm to correct the preliminary random load data set to obtain a corrected random load data set; S5. Establish a causal correlation database covering the dynamic response relationship between the vehicle and the bridge through the corrected random load data set, the preliminary calibration data set and the second calibration data set; S6. Construct a physical information neural network model, train the physical information neural network using a supervised learning method based on a causal association database, optimize network parameters, and obtain a trained physical information neural network model; S7. Collect bridge dynamic response data in real time and input it into the trained physical information neural network model to conduct online evaluation of the bridge operation status, check abnormal responses and generate early warning signals.

[0025] Specifically, in one embodiment of the present invention, the bridge dynamic displacement acquisition equipment includes a digital image correlation acquisition device deployed under the bridge, which has a multi-scale image acquisition unit and a real-time optical flow algorithm module for non-contact monitoring of the deflection changes of key parts of the bridge under different frequency loads; the trajectory visual acquisition equipment includes a high-resolution camera array and a multi-view synchronous collector deployed on the bridge deck and the bridge head. The camera array obtains vehicle shape, speed and driving trajectory information through the Yolov8 target detection model, and fuses these visual data with the data output by the bridge dynamic displacement acquisition equipment on the same time basis.

[0026] Specifically, the bridge dynamic displacement acquisition equipment is deployed at the key parts of the bridge to monitor the displacement changes of the bridge under different load conditions in a non-contact manner. Among them: Digital Image Correlation (DIC) is selected as the dynamic displacement acquisition equipment. According to the structural characteristics and force distribution of the bridge, the multi-scale image acquisition units are evenly arranged at the key nodes under the bridge to ensure that the main stress-bearing parts of the bridge are covered. Each image acquisition unit is equipped with a high-resolution camera, which can acquire images of the bridge surface at different magnifications to achieve accurate monitoring of the bridge's subtle displacement. The real-time optical flow algorithm module is integrated in the dynamic displacement acquisition device to process the captured image sequence and calculate the displacement of each monitoring point of the bridge in real time.

[0027] The track vision acquisition device is used to monitor the dynamic information of vehicles on the bridge, including the shape, speed and driving trajectory of the vehicle. Specifically, multiple high-resolution cameras are installed on the bridge deck and the bridge head to form a camera array. The camera array should cover the entire bridge deck to ensure comprehensive monitoring of all moving vehicles. The camera should have a wide-angle lens and a high frame rate to capture images of fast-moving vehicles. Multi-Angle Synchronized Collectors are set between each camera to achieve synchronous data acquisition from different perspectives. The Yolo v8 target detection model is embedded in the track vision acquisition device to process the collected vehicle images in real time. The data collected by the bridge dynamic displacement acquisition device and the track vision acquisition device are synchronized and fused according to a unified time reference. Ensure the consistency of the bridge structure response data and the vehicle load data in time and space.

[0028] Specifically, in one embodiment of the present invention, step S2 includes: Select a test vehicle that meets industry standards, such as a standardized truck or heavy-duty vehicle, to ensure that its load characteristics are consistent with actual traffic vehicles. Determine the static and dynamic load parameters of the vehicle, including total mass, axle weight distribution, vehicle size, and tire pressure. Equip the necessary sensor equipment, such as load sensors and position sensors, to monitor the actual load and driving status of the vehicle in real time.

[0029] According to the structural characteristics of the bridge and the results of the force analysis, plan the vehicle's driving path on the bridge to ensure that the main stress-bearing parts of the bridge are covered. Set the vehicle's driving speed and acceleration to simulate different traffic flows and dynamic load conditions when vehicles pass through. Determine the spacing and number of repetitions of vehicle driving to ensure the adequacy and representativeness of data collection.

[0030] According to the implementation method in step S1, the bridge dynamic displacement acquisition equipment and trajectory visual acquisition equipment are correctly installed. Through preliminary tests under known load conditions, the visual data acquisition system is calibrated, and the focal length, exposure parameters and image processing algorithm parameters of the camera are adjusted. Data synchronization tests are performed to ensure the synchronization of the bridge dynamic response data and the vehicle trajectory data in time and space.

[0031] Set up a standardized load vehicle on the bridge to ensure that the vehicle status meets the preset load parameters and driving trajectory. Start the visual data acquisition system to record the dynamic response data generated by the vehicle during driving on the bridge, including parameters such as bridge displacement, stress and vibration. Record and store the dynamic response data of the bridge under different load conditions in real time. According to the design requirements, repeat multiple calibration tests, covering different load conditions and driving speeds, to obtain diversified calibration data.

[0032] Filter out the noise and abnormal data that may exist during the acquisition process. Annotate the acquired dynamic response data and record the corresponding load parameters, vehicle type, driving status and other information. Store the processed calibration data in a unified format in the preliminary calibration data set.

[0033] Specifically, in one embodiment of the present invention, step S3 includes: Multiple high-resolution cameras are arranged on the bridge deck and at both ends, and the Yolo v8 target detection model is used to process the real-time collected images to accurately identify and classify different types of vehicles (such as small cars, trucks, heavy vehicles, etc.). Subsequently, a multi-target tracking algorithm (such as the SORT algorithm) is applied to track the identified vehicles and record their dynamic parameters such as the driving path, speed, and acceleration on the bridge.

[0034] Based on the output of the Yolo v8 model, a deep learning classification algorithm is further applied to finely classify vehicles. The classification model is trained to identify the type, size, and load characteristics of the vehicle. The collected data is manually annotated and verified. For incorrectly identified vehicle types, secondary inspection or manual correction is performed.

[0035] The DIC device deployed at key parts of the bridge monitors the displacement changes of the bridge under natural traffic loads in real time through a multi-scale image acquisition unit. The DIC device can accurately capture the tiny displacement of the bridge under different frequency load conditions and record key parameters such as deflection, stress and vibration amplitude. The real-time optical flow algorithm module integrated in the DIC device processes the collected image sequence and calculates the displacement of each monitoring point on the bridge.

[0036] The vehicle type, trajectory parameters and bridge response data are associated and annotated to form a structured data record. Each record contains multi-dimensional parameters such as vehicle type, driving speed, acceleration, bridge displacement, stress and vibration. The processed data is stored in a preliminary random load data set in a unified format (such as CSV, JSON or database table).

[0037] Specifically, in one embodiment of the present invention, step S4 includes: To ensure data consistency and comparability, select the same type of standardized specific load vehicle as in step S2. And the same vehicle driving trajectory as in step S2. Set up the standardized load vehicle on the bridge, start the visual data acquisition system, and record the dynamic response data generated by the vehicle during driving on the bridge, including parameters such as bridge displacement, stress, and vibration. Record and store the dynamic response data of the bridge under different load conditions in real time. According to the design requirements, repeat the calibration test multiple times, covering the same load conditions and driving speeds, to obtain the second calibration data set.

[0038] By comparing the initial calibration data set with the second calibration data set, systematic errors and random errors are identified. Systematic errors: Analyze the consistency of the bridge response data in the two calibration tests to identify systematic deviations caused by equipment installation, changes in environmental conditions, etc. Random errors: Identify random fluctuations and noise in the two calibration data through statistical analysis to evaluate random uncertainties in the data acquisition process.

[0039] Based on the identified systematic errors and random errors, the error correction algorithm is applied to correct the preliminary random load data set. The specific steps include: Step 1: Define the initial error vector ,in is the initial system error, is the initial random error, is the initial environmental error; Step 2: Calculate the intermediate correction function at the kth iteration correction ,in, For the preliminary calibration dataset, is the second calibration data set, is the adaptive convergence factor, Based on multidimensional nonlinear mapping, the previous iteration result and , After coupling operation, a new correction vector is output ; Step 3: Substitute the intermediate correction result into the correction formula: , In the formula, is the exponential penalty factor, represents the vector norm, It is a quadratic coupling correction function, which performs nonlinear compensation according to the distribution characteristics of vehicle load effects and environmental disturbance characteristics; Step 4: After several iterations, when or Output the final correction result when it is lower than the predetermined threshold , and Applied to the preliminary random load data set, a revised random load data set is obtained.

[0040] The present invention accurately identifies systematic errors and random errors by performing a second known load calibration test and comparing the initial calibration data set with the second calibration data set. Iterative optimization is performed using a quadratic coupling correction function of multidimensional nonlinear mapping to effectively eliminate systematic deviations and random fluctuations generated during data acquisition.

[0041] Specifically, in one embodiment of the present invention, step S5 includes: S51, integrating the corrected random load data set, the preliminary calibration data set and the second calibration data set, associating the vehicle load parameters with the bridge response parameters according to the timestamp, and obtaining a synchronous record of multi-source data.

[0042] Specifically, the corrected random load data set, the preliminary calibration data set, and the second calibration data set are merged to form a unified data source. Based on a unified time base, the vehicle load parameters (such as vehicle type, speed, trajectory) and the bridge response parameters (such as displacement, stress, vibration) are accurately matched according to the timestamp to form a synchronized multi-source data record. The time synchronization mechanism is used to ensure the consistency of data from different data sources on the time axis. Data synchronization tools are used to process and transmit data in real time. The synchronized data records are stored in a data warehouse according to a unified structure, and multi-source data are managed using a relational database or a time series database.

[0043] S52. Perform multi-level feature extraction on the above-mentioned synchronous records to obtain a data layer, a feature layer and an association layer respectively; wherein the data layer contains the original data of the vehicle node, the response node and the environment node, the feature layer includes frequency domain and time domain features, and the association layer is used to record the direct or indirect causal relationship between the nodes.

[0044] Specifically, the data structure is divided into data layer, feature layer and association layer. The data layer is the original data from vehicle nodes, response nodes and environment nodes, and the data is divided into vehicle nodes (recording vehicle load parameters), response nodes (recording bridge dynamic response parameters) and environment nodes (recording environmental influencing factors such as temperature and humidity). The feature layer is the time domain features and frequency domain features, including statistical features: calculating statistical quantities such as mean, variance, kurtosis, skewness, etc. Dynamic features: extracting dynamic parameters such as maximum value, minimum value, amplitude, etc. in the displacement time series. Fourier transform: Fourier transform the time domain signal to extract the main frequency components and amplitude in the spectrum. Power spectral density: calculate the power spectral density of the signal and analyze the frequency distribution characteristics. The association layer includes: using statistical methods and machine learning algorithms (such as random forests) to identify the direct or indirect causal relationship between vehicle load and bridge response. Record the direct and indirect associations between nodes, build a multi-level causal relationship network, and reflect the complex interactions between different nodes.

[0045] S53. Extract the main causal chains based on causal strength to form direct and indirect causal relationships between vehicle loads and bridge responses; use optimization algorithms to optimize the accuracy and completeness of the causal chains.

[0046] The formula for calculating causal strength is: , In the formula, represents the causal strength from node i to node j; is the mutual information between node i and node j; is the directional consistency from node i to node j; is a nonlinear parameter for adjusting the causal strength.

[0047] Based on the calculated causal strength, the main causal chains, i.e., those node pairs with higher causal strength, are extracted to form the direct and indirect causal relationships between vehicle loads and bridge responses.

[0048] The Bayesian optimization algorithm is used to adjust the weight of the extracted causal chain. The formula is as follows: , in, is the final causal weight from node i to node j; m is the total number of nodes that have a causal relationship with node i.

[0049] Dynamically adjust the weight parameters in the causal chain through the Bayesian optimization algorithm , improving the accuracy and completeness of the causal chain.

[0050] S54. Enter the constructed causal chain and association relationship into the graph database to form a causal association database covering a multi-level data structure.

[0051] Select a graph database suitable for storing complex causal relationships, define nodes (vehicle nodes, response nodes, environment nodes) and edges (causal relationship edges) in the graph database, and assign corresponding attributes (such as causal strength, weight). Build a multi-level data structure covering the data layer, feature layer, and association layer. Use the optimized main causal chain as the core part of the graph database to record the direct and indirect impact paths of vehicle loads on bridge response.

[0052] Specifically, in one embodiment of the present invention, step S6 includes: A physical information neural network model is constructed. The model adopts a multi-branch hybrid structure. The input data of the model consists of two parts: vehicle load parameters: including vehicle type, weight, speed, passing trajectory, etc.; bridge dynamic response data: including displacement, stress, vibration, etc. Figure 2 As shown, the model structure includes: Physical branch: It includes an input layer, two fully connected hidden layers, and an output layer. Each hidden layer applies a physical constraint activation function. The physical branch is used to process the embedded bridge mechanics equations and output physical constraint results.

[0053] Specifically, the input layer of the physics branch receives the parameters of the bridge mechanics model (such as material properties and structural construction), and uses the BN layer for feature normalization. The first hidden layer uses the LeakyReLU activation function, initialized with He, followed by a batch normalization layer. During training, a Dropout layer is added, with the deactivation rate set to 0.3 and the number of nodes set to h1, depending on the number of input features; the second hidden layer uses the ReLU activation function and the number of nodes is set to h2. Each hidden layer is finally connected to a physical constraint layer, which sets a custom physical constraint activation function. The specific physical constraints include static constraints and dynamic constraints, where: The static constraint formula is: , In the formula, x is the spatial coordinate position, that is, the position parameter of each measuring point in the bridge; represents the mass matrix of the bridge; represents the damping matrix; represents the stiffness matrix; represents the displacement vector of the bridge; is the external load vector; express Second-order partial derivative with respect to time; express First-order partial derivative with respect to time; The dynamic constraint formula is: , In the formula, represents the natural frequency of the bridge; represents the vibration mode vector of the bridge; is the stiffness matrix.

[0054] Data branch: includes input layer, multi-scale convolution layer, attention mechanism layer and fully connected hidden layer; the multi-scale convolution layer is used to extract multi-scale features; the attention mechanism layer is used to autonomously learn the importance of different features; the data branch is used to process high-frequency visual data and extract deep feature information.

[0055] Specifically, the data branches include, in order of data flow: The input layer receives visual data, i.e., dynamic response data.

[0056] The multi-scale convolution module has small-scale convolution branches, medium-scale convolution branches, and large-scale convolution branches. The convolution kernel of the small-scale convolution branch is 3*3, the step size is 1, the activation function is ReLU, and it is followed by a BN layer; the convolution kernel of the medium-scale convolution branch is 5*5, the step size is 1, the activation function is ReLU, and it is followed by a BN layer; the convolution kernel of the large-scale convolution branch is 7*7, the step size is 1, the activation function is ReLU, and it is followed by a BN layer. Finally, there is a fusion layer to splice the output features of different scales in the channel dimension.

[0057] The attention mechanism layer is divided into channel attention and spatial attention; the spatial attention has two layers of 1*1 convolution, using Sigmoid activation, and its output is multiplied by the input; the channel attention includes global average pooling and 3 layers of fully connected layers, using Sigmoid activation, and its output is multiplied by the input.

[0058] The maximum pooling layer has a pooling kernel of 2*2 and a stride of 2.

[0059] Global average pooling layer.

[0060] The fully connected layer uses the ReLU activation function.

[0061] Fusion layer: Contains an adaptive weight fusion unit, which dynamically weights and fuses the outputs of the physical branch and the data branch; the fusion layer achieves effective integration of information by learning weight parameters.

[0062] Specifically, the physical branch output and the data branch output are first concatenated, and then two weight coefficients α and β are output through two fully connected layers FC1 and FC2. FC1: 128→64, ReLU activation; FC2: 64→2, Softmax activation. The weight coefficients α and β are used to dynamically weight the physical branch output and the data branch output. Output layer: integrates the fusion results of the physical branch and the data branch, and outputs the bridge operation status assessment.

[0063] The output layer contains two fully connected layers, fully connected layer 1 is used for feature integration, and fully connected layer 2 is used for prediction.

[0064] The model training process is as follows: The dataset from the causal association database is divided into a training set, a validation set, and a test set, with a ratio of 70% training and 30% validation. The dataset includes vehicle load parameters, bridge dynamic response parameters, and related environmental factors. A supervised learning method is used to use the labeled bridge health status data (such as normal, slightly abnormal, obviously abnormal, and seriously abnormal) as a supervisory signal to guide the model to learn the relationship between the dynamic response and health status of the bridge. The gradient descent method and its variants (such as the Adam optimizer) are used to optimize the network parameters and minimize the loss function. The loss function is designed as follows: , , , , In the formula, is the total loss function; For data driven losses; Loss of physical restraint; is the regularization loss; , , is the adaptive weight coefficient; represents the sth sample; is the number of samples; represents the actual value of the sample, that is, the real response data of the bridge at the sth position; Represents the model prediction value, that is, the bridge response data predicted by the model; To adjust the parameters; , , , , is the physical parameter corresponding to sample s; is the standardized parameter; is the regularization coefficient; For the model Layer weight parameter; Q is the number of network layers.

[0065] The model training steps are as follows: Initialization: Randomly initialize the network parameters, including the weights of the physical branch and the data branch.

[0066] Forward propagation: input training data, pass through the physical branch and data branch, and output the prediction result through the fusion layer.

[0067] Calculate loss: According to the actual output and predicted output, calculate the data-driven loss, physical constraint loss and regularization loss, and get the total loss .

[0068] Back propagation: The gradient of each parameter is calculated through the back propagation algorithm.

[0069] Parameter update: Use optimization algorithms to update network parameters to minimize the total loss.

[0070] Iterative training: Repeat forward propagation, loss calculation, back propagation, and parameter update until the loss function converges or the preset training rounds are reached.

[0071] Model validation: Evaluate model performance on the validation set and adjust the model structure or training parameters to improve the generalization ability of the model.

[0072] The physical information neural network model provided by the present invention effectively integrates the bridge mechanical equations and high-frequency visual data features, which not only improves the accuracy of bridge operation status assessment, but also enhances the generalization ability and robustness of the model.

[0073] Specifically, in one embodiment of the present invention, step S7 includes: S71. The dynamic response data generated by the bridge during operation is acquired in real time through the visual data acquisition system, and the acquired dynamic response data is time synchronized and preliminarily preprocessed.

[0074] Time synchronization is achieved using the NIP protocol and a unified time server, and a timestamp is added to each frame of collected data. Preliminary preprocessing includes Kalman filtering for denoising, linear interpolation to fill in missing data, and normalization of dynamic response data to unify data of different dimensions into the [0,1] interval.

[0075] S72. Convert the preprocessed dynamic response data into the input format required by the physical information neural network model, and input it into the trained physical information neural network model. Calculate the current operating status parameters of the bridge through the forward propagation process. The model outputs the health assessment results of the bridge according to the embedded physical constraints and causal relationship database, including the current deflection, vibration mode and potential structural abnormality indicators.

[0076] The vehicle load parameters (such as vehicle type, weight, speed, trajectory) collected in real time and the bridge dynamic response parameters (such as displacement, stress, vibration) are integrated into a unified feature vector. The data collected continuously within a time window are assembled into batches.

[0077] Load the trained physical information neural network model parameters, input the real-time collected dynamic response data into the model, and generate the current operating status parameters of the bridge through physical branch processing, data branch input, fusion layer integration and output layer evaluation, including deflection, vibration mode and potential structural abnormality indicators.

[0078] The model output refers to the causal relationship in the causal association database to ensure that the model output meets the physical constraints of the bridge mechanics equations.

[0079] Finally, the health assessment results are output, including quantitative parameters and structural indicators. Quantitative parameters refer to key operating parameters such as the current bridge deflection value and vibration amplitude. Structural indicators refer to evaluating the structural health status of the bridge and identifying potential structural anomalies (such as cracks, fatigue damage, etc.).

[0080] S73. The health assessment results based on the model output are compared with the preset safety standards and thresholds to determine whether the bridge operation status is normal, and a multi-level assessment mechanism is used to distinguish between minor abnormalities, obvious abnormalities and serious abnormalities.

[0081] According to the relevant industry bridge safety regulations, set safety thresholds for various operating parameters (such as maximum allowable deflection, vibration frequency, etc.). Optimize and adjust preset thresholds based on historical monitoring data and bridge design parameters.

[0082] If all key operating parameters (such as deflection and vibration amplitude) are lower than the preset safety threshold, the bridge is judged to be in normal operating condition.

[0083] Otherwise, an exception is determined, specifically: Minor abnormality: Some parameters slightly exceed the safety threshold and require attention but no immediate action is required.

[0084] Obvious abnormality: Multiple parameters significantly exceed the safety threshold, and timely maintenance or reinforcement measures are required.

[0085] Severe abnormality: Key parameters are seriously exceeded, which may cause structural damage or failure of the bridge, and emergency shutdown and repair measures are required.

[0086] S74. Generate warning signals of corresponding levels based on the abnormality assessment results, including level 1 warning, level 2 warning and level 3 warning, which correspond to slight abnormalities, obvious abnormalities and severe abnormalities respectively. Each level of warning signal contains specific abnormal parameter values, abnormality categories and recommended response measures. The warning signal is transmitted in real time to the bridge management and operation and maintenance system through wireless communication or wired network.

[0087] Level 1, level 2 and level 3 warnings are set according to minor abnormalities, obvious abnormalities and serious abnormalities. The classification basis includes the degree of parameter exceeding the limit, the duration of abnormality and the comprehensive evaluation of multiple parameters. After the level is determined, a warning signal is generated. The content of the warning signal includes: Abnormal parameter value: specifically list the operating parameters that exceed the standard and their actual values. Abnormal category: clearly marked as minor abnormality, obvious abnormality or serious abnormality. Recommended response measures: according to the warning level, give corresponding maintenance or repair suggestions. The warning signal format uses JSON format, including timestamp, bridge ID, abnormal category, abnormal parameters and recommended measures.

[0088] The warning signal is transmitted to the bridge management and operation and maintenance system by wireless or wired means.

[0089] In addition, if Figure 3 As shown, the present invention also provides a bridge warning system based on physical information neural network and machine vision, the system is used to execute any of the above methods, and the system includes: The data acquisition module integrates bridge dynamic displacement acquisition equipment and trajectory visual acquisition equipment to collect dynamic response data of the bridge and vehicle trajectory data in real time; The dual calibration module performs at least two known load calibration tests and natural traffic flow collection, evaluates the difference of random load data and calls the error correction algorithm for dynamic correction; The causal relationship database module stores, builds causal relationships, and iteratively updates multi-source heterogeneous data based on a graph database structure containing vehicle nodes, bridge response nodes, and environment nodes; Physical information neural network module, including a physical information neural network model with multi-branch hybrid structure, to evaluate the bridge operation status; The real-time evaluation and early warning module inputs dynamic response data into the trained neural network model in real time; generates multi-level early warning signals according to the model output, and transmits and feedbacks the signals through the early warning setting unit; Communication and storage module, used to transmit warning signals to the bridge management and operation and maintenance system; The user interface and monitoring module provides a visual bridge operation status monitoring interface; displays real-time data, warning information and historical records.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A bridge early warning method based on physical information neural network and machine vision, characterized in that: include: S1. Setting up a visual data acquisition system on the bridge, including bridge dynamic displacement acquisition equipment and trajectory visual acquisition equipment; S2. Design a known load calibration test, including selecting a standardized specific load vehicle and designing a predetermined trajectory. The visual data acquisition system collects dynamic response data under known load conditions to form a preliminary calibration data set; S3, collecting vehicle trajectories, vehicle types and corresponding bridge response data under natural traffic flow to form a preliminary random load data set; S4, conduct a known load calibration test again, collect the dynamic response of the bridge by the visual data acquisition system, form a second calibration data set, compare the preliminary calibration data set with the second calibration data set, identify systematic errors and random errors, and use the error correction algorithm to correct the preliminary random load data set to obtain a corrected random load data set; S5. Establish a causal correlation database covering the dynamic response relationship between the vehicle and the bridge through the corrected random load data set, the preliminary calibration data set and the second calibration data set; S6. Construct a physical information neural network model, train the physical information neural network using a supervised learning method based on a causal association database, optimize network parameters, and obtain a trained physical information neural network model; S7. Collect bridge dynamic response data in real time and input it into the trained physical information neural network model to conduct online evaluation of the bridge operation status, check abnormal responses and generate early warning signals.

2. The bridge early warning method based on physical information neural network and machine vision as claimed in claim 1 is characterized in that: The bridge dynamic displacement acquisition equipment includes a digital image correlation acquisition device deployed under the bridge, which has a multi-scale image acquisition unit and a real-time optical flow algorithm module for non-contact monitoring of the deflection changes of key parts of the bridge under different frequency loads; the trajectory visual acquisition equipment includes a high-resolution camera array and a multi-view synchronous collector deployed on the bridge deck and bridge head. The camera array obtains vehicle shape, speed and driving trajectory information through the Yolo v8 target detection model, and fuses these visual data with the data output by the bridge dynamic displacement acquisition equipment on the same time basis.

3. The bridge early warning method based on physical information neural network and machine vision as claimed in claim 1 is characterized in that: The error correction algorithm in step S4 is as follows: Step 1: Define the initial error vector ,in is the initial system error, is the initial random error, is the initial environmental error; Step 2: Calculate the intermediate correction function at the kth iteration correction ,in, For the preliminary calibration dataset, is the second calibration data set, is the adaptive convergence factor, Based on multidimensional nonlinear mapping, the previous iteration result and , After coupling operation, a new correction vector is output ; Step 3: Substitute the intermediate correction result into the correction formula: ， In the formula, is the exponential penalty factor, represents the vector norm, It is a quadratic coupling correction function, which performs nonlinear compensation according to the distribution characteristics of vehicle load effects and environmental disturbance characteristics; Step 4: After several iterations, when or Output the final correction result when it is lower than the predetermined threshold , and Applied to the preliminary random load data set, a revised random load data set is obtained.

4. The bridge early warning method based on physical information neural network and machine vision as claimed in claim 1 is characterized in that: Step S5 includes: S51, integrating the corrected random load data set, the preliminary calibration data set and the second calibration data set, associating the vehicle load parameters with the bridge response parameters according to the timestamps, and obtaining a synchronous record of multi-source data; S52, performing multi-level feature extraction on the above synchronous records to obtain a data layer, a feature layer and an association layer respectively; wherein the data layer includes the original data of the vehicle node, the response node and the environment node, the feature layer includes frequency domain and time domain features, and the association layer is used to record the direct or indirect causal relationship between the nodes; S53. Extract the main causal chain based on the causal strength to form the direct and indirect causal relationship between vehicle load and bridge response; use optimization algorithm to optimize the accuracy and completeness of the causal chain; S54. Enter the constructed causal chain and association relationship into the graph database to form a causal association database covering a multi-level data structure.

5. The bridge early warning method based on physical information neural network and machine vision as claimed in claim 4 is characterized in that: The formula for calculating causal strength is: ， In the formula, represents the causal strength from node i to node j; is the mutual information between node i and node j; is the directional consistency from node i to node j; A nonlinear parameter to adjust the causal strength; The Bayesian optimization algorithm is used to adjust the weight of the extracted causal chain, and the formula is defined as: ， in, is the final causal weight from node i to node j; m is the total number of nodes that have a causal relationship with node i.

6. The bridge early warning method based on physical information neural network and machine vision as claimed in claim 1, characterized in that: The physical information neural network model is a multi-branch hybrid structure, including: Physical branch: It includes an input layer, two fully connected hidden layers, and an output layer. Each hidden layer applies a physical constraint activation function. The physical branch is used to process the embedded bridge mechanics equations and output the physical constraint results. Data branch: includes input layer, multi-scale convolution layer, attention mechanism layer and fully connected hidden layer; multi-scale convolution layer is used to extract multi-scale features; attention mechanism layer is used to autonomously learn the importance of different features; data branch is used to process high-frequency visual data and extract deep feature information; Fusion layer: contains an adaptive weight fusion unit, which dynamically weights and fuses the outputs of the physical branch and the data branch. The fusion layer achieves effective information integration by learning weight parameters. Output layer: integrates the fusion results of the physical branch and the data branch, and outputs the bridge operation status assessment.

7. The bridge early warning method based on physical information neural network and machine vision as claimed in claim 6, characterized in that: The bridge mechanics equations in the physics branch include static constraints and dynamic constraints, among which: The static constraint formula is: ， In the formula, x is the spatial coordinate position, that is, the position parameter of each measuring point in the bridge; represents the mass matrix of the bridge; represents the damping matrix; represents the stiffness matrix; represents the displacement vector of the bridge; is the external load vector; express Second-order partial derivative with respect to time; express First-order partial derivative with respect to time; The dynamic constraint formula is: ， In the formula, represents the natural frequency of the bridge; represents the vibration mode vector of the bridge; is the stiffness matrix.

8. The bridge early warning method based on physical information neural network and machine vision as claimed in claim 7, characterized in that: The loss function used in training the physical information neural network model is: ， ， ， ， In the formula, is the total loss function; For data driven losses; Loss of physical restraint; is the regularization loss; , , is the adaptive weight coefficient; represents the sth sample; is the number of samples; represents the actual value of the sample, that is, the real response data of the bridge at the sth position; Represents the model prediction value, that is, the bridge response data predicted by the model; To adjust the parameters; , , , , is the physical parameter corresponding to sample s; is the standardized parameter; is the regularization coefficient; For the model Layer weight parameter; Q is the number of network layers.

9. The bridge early warning method based on physical information neural network and machine vision as claimed in claim 2, characterized in that: Step S7 includes: S71, acquiring dynamic response data generated by the bridge during operation in real time through a visual data acquisition system, and performing time synchronization and preliminary preprocessing on the acquired dynamic response data; S72, converting the preprocessed dynamic response data into the input format required by the physical information neural network model, and inputting the data into the trained physical information neural network model, calculating the current operating state parameters of the bridge through the forward propagation process, and the model outputs the health assessment results of the bridge according to the embedded physical constraints and causal association database, including the current deflection, vibration mode and potential structural abnormality indicators; S73. Based on the health assessment results output by the model, the preset safety standards and thresholds are compared to determine whether the bridge operation status is normal, and a multi-level assessment mechanism is used to distinguish between slight abnormalities, obvious abnormalities and serious abnormalities; S74. Generate warning signals of corresponding levels based on the abnormality assessment results, including level 1 warning, level 2 warning and level 3 warning, which correspond to slight abnormalities, obvious abnormalities and severe abnormalities respectively. Each level of warning signal contains specific abnormal parameter values, abnormality categories and recommended response measures. The warning signal is transmitted in real time to the bridge management and operation and maintenance system through wireless communication or wired network.

10. The bridge warning system based on physical information neural network and machine vision is characterized by: The system is used to perform the method according to any one of claims 1 to 9, and the system comprises: The data acquisition module integrates the bridge dynamic displacement acquisition equipment and the trajectory visual acquisition equipment to collect the dynamic response data of the bridge and the vehicle trajectory data in real time; The dual calibration module performs at least two known load calibration tests and natural traffic flow collection, evaluates the difference of random load data and calls the error correction algorithm for dynamic correction; The causal relationship database module stores, builds causal relationships, and iteratively updates multi-source heterogeneous data based on a graph database structure containing vehicle nodes, bridge response nodes, and environment nodes; Physical information neural network module, including a physical information neural network model with multi-branch hybrid structure, to evaluate the bridge operation status; The real-time evaluation and early warning module inputs dynamic response data into the trained neural network model in real time; generates multi-level early warning signals according to the model output, and transmits and feedbacks the signals through the early warning setting unit; Communication and storage module, used to transmit warning signals to the bridge management and operation and maintenance system; The user interface and monitoring module provides a visual bridge operation status monitoring interface; displays real-time data, warning information and historical records.

Images