Bridge collapse early warning method and system based on bridge image and neural network

Abstract

The application discloses a bridge collapse early warning method and system based on a bridge image and a neural network. The method comprises the following steps: acquiring a bridge image of a bridge prone to collapse; performing pretreatment and feature extraction on the bridge image to obtain bridge high-level features containing bridge structures and vehicle distribution; inputting the bridge high-level features containing the bridge structures and the vehicle distribution into a trained neural network model to output a prediction result of bridge collapse and vehicle drop; evaluating whether the bridge has a bridge collapse risk or a vehicle drop risk based on the prediction result of the bridge collapse; and performing bridge collapse early warning of a corresponding level according to the evaluation result of the bridge collapse risk or the vehicle drop risk. The application realizes bridge collapse prediction by extracting high-level features of a bridge image and combining a neural network, and can perform real-time early warning according to the predicted bridge collapse risk, thereby improving the accuracy of bridge collapse prediction and the real-time performance of early warning.

Description

Technical Field

[0001] This invention relates to the fields of big data and bridge inspection and management technology, specifically to a bridge collapse early warning method and system based on bridge images and neural networks. Background Technology

[0002] In modern infrastructure construction, bridges, as a key component of transportation networks, are crucial for safeguarding people's lives and property and promoting socio-economic development. With advancements in engineering technology and increasing traffic demands, bridge structures have become more complex and spanned longer, thus raising the requirements for their safety monitoring and emergency early warning systems. Although existing technologies enable regular inspections and real-time monitoring of bridges, achieving rapid and accurate early warning and emergency response in the face of sudden bridge collapses remains a challenge.

[0003] Traditional bridge monitoring methods primarily rely on physical detection equipment and periodic manual inspections, which have significant limitations in handling sudden bridge collapses. First, they typically cannot provide real-time warnings because data collection and analysis take time, hindering immediate response in emergencies. Second, even if anomalies are detected, effectively and promptly communicating warning information to relevant personnel and vehicles to prevent or mitigate secondary damage remains a technical challenge. Furthermore, with changing environmental factors and the increasing age of bridges, existing monitoring systems may fail to accurately predict and identify all potential collapse risks.

[0004] Therefore, designing a scheme that can effectively monitor and predict the risk of bridge collapse and issue corresponding collapse warnings is a practical problem that urgently needs to be solved. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the technical problem to be solved by this invention is: how to provide a bridge collapse early warning system based on bridge images and neural networks, which can predict bridge collapse by extracting high-level features from bridge images and combining them with neural networks, and can provide real-time early warnings based on the predicted bridge collapse risk, thereby improving the accuracy of bridge collapse prediction and the real-time nature of early warnings.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] Bridge collapse early warning methods based on bridge images and neural networks include:

[0008] S1: Determine the vulnerable locations of the bridge and obtain images of the bridge at these vulnerable locations;

[0009] S2: Preprocess the bridge image to obtain a preprocessed bridge image;

[0010] S3: Extract features from the preprocessed bridge image to obtain high-level bridge features that include bridge structure and vehicle distribution;

[0011] S4: Input the high-level features of the bridge, including the bridge structure and vehicle distribution, into the trained neural network model, and output the prediction results of bridge collapse and vehicle fall.

[0012] S5: Assess whether there is a risk of bridge collapse or vehicle falling off the bridge based on the prediction results of bridge collapse;

[0013] S6: Based on the assessment results of bridge collapse risk or vehicle falling risk, issue an early warning for bridge collapse of the corresponding level.

[0014] Preferably, in step S1, the bridge is subjected to structural mechanics analysis, vehicle and ship collision location analysis, and natural disaster vulnerability location analysis using finite element technology to identify potential weak points and collapse probability of the bridge, thereby rating the overall safety status of the bridge and determining the locations of the bridge prone to collapse.

[0015] Preferably, in step S2, the preprocessing of the bridge image includes image sharpening, noise removal, and contrast adjustment.

[0016] Preferably, in step S3, the preprocessed bridge image is used to extract features through the VGG19 pre-trained model, including bridge structural features such as overall bridge integrity, deformation or damage signs, and vehicle distribution features such as the number and distribution of vehicles on the bridge, and the bridge structural features and vehicle distribution features are spliced ​​together to form high-level bridge features.

[0017] Preferably, in step S4, the neural network model includes a trained bridge detection sub-model and a vehicle detection sub-model.

[0018] The bridge detection sub-model predicts whether a bridge will collapse based on the extracted high-level bridge features, and outputs the corresponding bridge collapse prediction probability as the bridge collapse prediction result.

[0019] The vehicle detection sub-model performs vehicle target detection and vehicle fall classification based on the extracted high-level bridge features, and outputs the predicted vehicle bounding box and the probability of whether the vehicle has fallen as the prediction result of bridge fall.

[0020] Preferably, the training steps for the bridge detection sub-model are as follows:

[0021] S401: Obtain the VGG19 pre-trained model pre-trained on the ImageNet dataset;

[0022] S402: Obtain bridge images to be used as training data and standardize the training data according to the format of the ImageNet dataset;

[0023] S403: Input the standardized training data into the VGG19 pre-trained model and extract the high-level features of the bridge through its convolutional and pooling layers;

[0024] S404: Input the extracted high-level bridge features into the bridge detection sub-model built on the fully connected layer, and output the corresponding bridge collapse prediction probability through the bridge detection sub-model;

[0025] S405: Calculate the training loss of the bridge detection sub-model based on the actual values ​​of bridge collapse and the corresponding predicted probabilities of bridge collapse from the training data, and optimize the parameters of the bridge detection sub-model through backpropagation;

[0026] The loss function formula for the bridge detection sub-model is as follows:

[0027]

[0028] BCE(p,y)=-[y×log(p)-(1-y)×log(1-p)];

[0029] In the formula: BCE(p,y) represents the degree of difference between the true value y and the predicted value p; Tal_BCE represents the average degree of difference between the true value and the predicted value of n training data; p i y represents the predicted value of the i-th training data; i This represents the true value of the i-th training data;

[0030] S406: Repeat steps S403 to S405 until the bridge detection sub-model converges.

[0031] Preferably, the training steps for the vehicle detection sub-model are as follows:

[0032] S411: Obtain the VGG19 pre-trained model pre-trained on the ImageNet dataset;

[0033] S412: Obtain bridge images to be used as training data and standardize the training data according to the format of the ImageNet dataset;

[0034] S413: Input the standardized training data into the VGG19 pre-trained model and extract the high-level features of the bridge through its convolutional and pooling layers;

[0035] S414: Input the extracted high-level bridge features into the vehicle detection sub-model built based on the YOLOv1 target detection algorithm. The vehicle detection sub-model performs vehicle target detection, divides the image into multiple grid cells, and predicts the vehicle bounding box and the probability of whether the vehicle has fallen off the grid cell for each grid cell. Then, non-maximum suppression is used to filter and remove redundant vehicle bounding boxes. Finally, the predicted vehicle bounding box and the probability of whether the vehicle has fallen off the grid cell are output.

[0036] S415: Based on the true annotation of the vehicle bounding box and the true probability of whether the vehicle has fallen off the training data, the training loss and R2 value of the vehicle detection sub-model are calculated by combining the predicted vehicle bounding box and the class probability of whether the vehicle has fallen off the training data, and the parameters of the vehicle detection sub-model are optimized by backpropagation.

[0037] S416: Repeat steps S413 to S415 until the vehicle detection sub-model converges.

[0038] Preferably, in step S5, the collapse warning includes:

[0039] 1) A sound and flashing light alarm will be activated to alert the user;

[0040] 2) Display warning information on electronic screens;

[0041] 3) Issue early warning information through social media platforms or public broadcasting systems;

[0042] 4) Adjust traffic flow and routes near the bridge by modifying the traffic signal control system;

[0043] 5) Establish a contact database in advance and send warning text messages or emails to the contacts.

[0044] Preferably, two collapse thresholds are set when assessing the risk of bridge collapse: if the predicted probability of bridge collapse is less than the first collapse threshold, there is no risk of bridge collapse; if the predicted probability of bridge collapse is greater than or equal to the first collapse threshold and less than the second collapse threshold, it is determined that there is a risk of bridge collapse; if the predicted probability of bridge collapse is greater than or equal to the second collapse threshold, it is determined that there is a risk of bridge collapse.

[0045] A drop threshold is set when assessing the risk of a vehicle falling off: if the class probability of the vehicle bounding box is less than the drop threshold, there is no risk of the vehicle falling off; if the class probability of the vehicle bounding box is greater than or equal to the drop threshold, it is determined that there is a risk of the vehicle falling off.

[0046] The tiered early warning logic for bridge collapse includes:

[0047] 1) Level 3 Orange Alert: It is determined that there is a risk of bridge collapse; sound and light alarms are issued and warning information is displayed on electronic screens;

[0048] 2) Level II Blue Alert: This alert indicates a risk of bridge collapse; it uses audible and visual sirens and flashing lights to alert the system, displays warning information on electronic screens, disseminates warning information through social media platforms or public address systems, and adjusts traffic flow and routes near the bridge by modifying traffic signal control systems.

[0049] 3) Level 1 Red Alert: If it is determined that there is a risk of bridge collapse and vehicle falling at the same time, sound and light alarms will be issued and warning information will be displayed on electronic screens; warning information will be issued through social media platforms or public broadcasting systems; traffic flow and driving routes near the bridge will be changed by adjusting traffic signal control systems; and warning text messages or emails will be sent to contacts.

[0050] This invention also discloses a bridge collapse early warning system based on bridge images and neural networks, which is implemented based on the bridge collapse early warning method based on bridge images and neural networks in this invention, specifically including:

[0051] Monitoring equipment is used to acquire images of bridges in locations prone to collapse.

[0052] A custom comprehensive analysis module is used to first preprocess the bridge image to obtain a preprocessed bridge image; secondly, feature extraction is performed on the preprocessed bridge image to obtain high-level bridge features including bridge structure and vehicle distribution; then, the high-level bridge features including bridge structure and vehicle distribution are input into a trained neural network model to output prediction results of bridge collapse and vehicle fall; finally, based on the prediction results of bridge collapse, the module assesses whether there is a risk of bridge collapse or vehicle fall.

[0053] The information storage module is used to store bridge images, as well as assessment results of the risk of bridge collapse, or whether the bridge has already collapsed or a vehicle has fallen off.

[0054] The early warning device is used to issue corresponding levels of collapse warnings based on the assessment results of bridge collapse risk and vehicle fall risk.

[0055] Compared with existing technologies, the bridge collapse early warning system based on bridge images and neural networks in this invention has the following advantages:

[0056] This invention acquires bridge images of locations prone to collapse and preprocesses these images. First, the invention pre-determines the locations of bridges prone to collapse and acquires targeted bridge images of these locations to enable subsequent bridge collapse prediction. This ensures the accuracy of bridge collapse prediction while reducing the amount of data processing and improving prediction efficiency. Second, preprocessing removes noise, blur, and unnecessary details from the bridge images, making the bridge structure and vehicle distribution clearer, thereby improving the accuracy of subsequent feature extraction. Furthermore, preprocessing ensures that bridge images from different sources, at different times, and under different conditions have a consistent format and quality, facilitating subsequent feature extraction and model training. Finally, preprocessing removes redundant information, reduces image size, lowers the computational complexity of subsequent processing, and ultimately improves the efficiency of bridge collapse prediction and processing.

[0057] This invention extracts features from preprocessed bridge images to obtain high-level bridge features that include bridge structure and vehicle distribution. Based on these extracted high-level features, it predicts bridge collapse and vehicle falls. First, the extracted high-level features, containing bridge structure and vehicle distribution, effectively describe the structural characteristics of the bridge and the distribution of vehicles, helping subsequent models accurately identify the state and potential risks of the bridge and vehicles on it. Second, by extracting high-level features, the high-dimensional data of the original bridge image can be converted into low-dimensional feature vectors, facilitating subsequent model processing and training. Finally, the high-level features extracted in this invention have stronger robustness and representativeness, enabling the model to better adapt to bridge images under different scenarios and conditions.

[0058] This invention, based on the extraction of high-level bridge features including bridge structure and vehicle distribution, further utilizes a neural network model to predict bridge collapse and vehicle falls. First, the neural network model can automatically learn and extract complex features from bridge images and optimize its prediction performance through training on a large amount of data, thus achieving efficient and accurate predictions of bridge collapse and vehicle falls. Second, the neural network model can handle complex nonlinear relationships, making it suitable for complex prediction tasks such as bridge collapse and vehicle falls, which are influenced by multiple factors, thus ensuring the effectiveness of bridge collapse prediction. Finally, the neural network model can self-adjust and optimize based on new data and circumstances, continuously improving its prediction performance and further enhancing the accuracy of bridge collapse prediction.

[0059] This invention assesses the risk of bridge collapse or vehicle fall based on bridge collapse predictions and issues corresponding levels of early warning. First, by assessing the risks of bridge collapse and vehicle fall, and establishing corresponding warning levels, real-time early warning of bridge collapse is achieved, enhancing emergency response capabilities. It can immediately alert surrounding vehicles and pedestrians upon detecting a collapse risk, providing a crucial time window to prevent secondary falls or reduce damage from accidents, thus effectively mitigating the safety risks posed by bridge collapse and vehicle fall. Second, issuing different levels of warning signals based on different risk levels and actual conditions helps management personnel and those on the bridge determine the bridge's collapse status based on the warning information, allowing for appropriate countermeasures and improving the effectiveness of bridge collapse early warning. Attached Figure Description

[0060] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0061] Figure 1 A logic diagram for a bridge collapse alert method;

[0062] Figure 2 , Figure 3 , Figure 4 , Figure 5 Instance graphs of the training data;

[0063] Figure 6 The training flowchart for the bridge detection sub-model;

[0064] Figure 7 A flowchart illustrating the workflow of the vehicle detection sub-model;

[0065] Figure 8 and Figure 9 The logic block diagram and workflow diagram for the bridge collapse warning system. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0067] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the figures, or the orientation or positional relationship commonly used when the product is in use. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. In addition, the terms "horizontal," "vertical," etc., do not mean that the component is required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0068] The following detailed explanation illustrates the specific implementation methods:

[0069] Example:

[0070] This embodiment discloses a bridge collapse early warning method based on bridge images and neural networks.

[0071] like Figure 1 As shown, the bridge collapse early warning method based on bridge images and neural networks includes:

[0072] S1: Determine the vulnerable locations of the bridge and obtain images of the bridge at these vulnerable locations;

[0073] S2: Preprocess the bridge image to obtain a preprocessed bridge image;

[0074] S3: Extract features from the preprocessed bridge image to obtain high-level bridge features that include bridge structure and vehicle distribution;

[0075] S4: Input the high-level features of the bridge, including the bridge structure and vehicle distribution, into the trained neural network model, and output the prediction results of bridge collapse and vehicle fall.

[0076] S5: Assess whether there is a risk of bridge collapse or vehicle falling off the bridge based on the prediction results of bridge collapse;

[0077] S6: Based on the assessment results of bridge collapse risk or vehicle falling risk, issue an early warning for bridge collapse of the corresponding level.

[0078] This invention acquires bridge images of locations prone to collapse and preprocesses these images. First, the invention pre-determines the locations of bridges prone to collapse and acquires targeted bridge images of these locations to enable subsequent bridge collapse prediction. This ensures the accuracy of bridge collapse prediction while reducing the amount of data processing and improving prediction efficiency. Second, preprocessing removes noise, blur, and unnecessary details from the bridge images, making the bridge structure and vehicle distribution clearer, thereby improving the accuracy of subsequent feature extraction. Furthermore, preprocessing ensures that bridge images from different sources, at different times, and under different conditions have a consistent format and quality, facilitating subsequent feature extraction and model training. Finally, preprocessing removes redundant information, reduces image size, lowers the computational complexity of subsequent processing, and ultimately improves the efficiency of bridge collapse prediction and processing.

[0079] This invention extracts features from preprocessed bridge images to obtain high-level bridge features that include bridge structure and vehicle distribution. Based on these extracted high-level features, it predicts bridge collapse and vehicle falls. First, the extracted high-level features, containing bridge structure and vehicle distribution, effectively describe the structural characteristics of the bridge and the distribution of vehicles, helping subsequent models accurately identify the state and potential risks of the bridge and vehicles on it. Second, by extracting high-level features, the high-dimensional data of the original bridge image can be converted into low-dimensional feature vectors, facilitating subsequent model processing and training. Finally, the high-level features extracted in this invention have stronger robustness and representativeness, enabling the model to better adapt to bridge images under different scenarios and conditions.

[0080] This invention, based on the extraction of high-level bridge features including bridge structure and vehicle distribution, further utilizes a neural network model to predict bridge collapse and vehicle falls. First, the neural network model can automatically learn and extract complex features from bridge images and optimize its prediction performance through training on a large amount of data, thus achieving efficient and accurate predictions of bridge collapse and vehicle falls. Second, the neural network model can handle complex nonlinear relationships, making it suitable for complex prediction tasks such as bridge collapse and vehicle falls, which are influenced by multiple factors, thus ensuring the effectiveness of bridge collapse prediction. Finally, the neural network model can self-adjust and optimize based on new data and circumstances, continuously improving its prediction performance and further enhancing the accuracy of bridge collapse prediction.

[0081] This invention assesses the risk of bridge collapse or vehicle fall based on bridge collapse predictions and issues corresponding levels of early warning. First, by assessing the risks of bridge collapse and vehicle fall, and establishing corresponding warning levels, real-time early warning of bridge collapse is achieved, enhancing emergency response capabilities. It can immediately alert surrounding vehicles and pedestrians upon detecting a collapse risk, providing a crucial time window to prevent secondary falls or reduce damage from accidents, thus effectively mitigating the safety risks posed by bridge collapse and vehicle fall. Second, issuing different levels of warning signals based on different risk levels and actual conditions helps management personnel and those on the bridge determine the bridge's collapse status based on the warning information, allowing for appropriate countermeasures and improving the effectiveness of bridge collapse early warning.

[0082] In its specific implementation, this invention uses finite element technology to perform structural mechanics analysis, vehicle and ship collision location analysis, and natural disaster vulnerability location analysis on bridges. It identifies potential weak points and collapse possibilities of bridges, and then rates the overall safety status of bridges and determines the locations where bridges are prone to collapse.

[0083] In this embodiment, the location of the bridge prone to collapse is determined through the following logic:

[0084] 1) Based on the bridge design drawings provided by the bridge maintenance unit, a full-size, detailed solid finite element simulation model of the bridge structure was established according to its structural characteristics. The finite element simulation model was then corrected based on various damage indicators from bridge inspection reports over the years. Next, using the corrected finite element simulation model, analyses including ship-bridge collision, vehicle-bridge collision, scour effects, mudflow impact, and bridge durability were performed to obtain the maximum deformation and maximum stress of the main load-bearing structure under various working conditions.

[0085] 2) The safety status of each structure is scored based on the damage grading limits of displacement or stress. For example, for a T-beam, the main focus is on its transverse or longitudinal displacement. If the displacement of the T-beam obtained through finite element analysis is greater than the normal function of its fixed or sliding supports but the beam does not fall, a damage score of 50 is given; if the displacement is greater than the limit for falling, a damage score of 100 is given.

[0086] 3) Calculate the damage score of all bridge components through step 2) above, and take the location of the component with the high damage score as the location of the bridge that is prone to collapse.

[0087] In practice, the preprocessing of bridge images includes image sharpening, noise removal, and contrast adjustment.

[0088] In this embodiment, the preprocessing of bridge images is achieved through the following methods:

[0089] 1) Image sharpening:

[0090] Sharpening filters: These filters enhance the edges and details of an image, making it sharper. This typically involves detecting and enhancing edges in the image.

[0091] Contrast enhancement: By increasing the contrast of an image, the details of a bridge can be made more prominent. There are various contrast enhancement methods, such as histogram equalization and adaptive histogram equalization.

[0092] 2) Noise Removal:

[0093] Mean filtering: This is a simple noise removal method that replaces the original pixel value by calculating the average value of each pixel and its surrounding pixels. This method is suitable for removing mild Gaussian noise.

[0094] Median filtering: Median filtering is an effective method for removing impulse noise such as salt-and-pepper noise. It replaces the original pixel value with the median value within the pixel's neighborhood.

[0095] Gaussian filtering: Gaussian filtering is a smoothing filter that can remove noise while preserving image details. It achieves this by applying a Gaussian blur to the image.

[0096] Transform domain filtering, such as Fourier transform or wavelet transform, can transform an image from the spatial domain to the transform domain, then process the transform coefficients to remove noise, and finally perform an inverse transform back to the spatial domain.

[0097] 3) Contrast adjustment:

[0098] Histogram adjustment: By adjusting the histogram distribution of an image, the image contrast can be changed. For example, stretching the histogram can improve the image contrast.

[0099] Contrast stretching: Enhances image contrast by expanding the dynamic range of gray levels in the image. This typically involves setting a minimum and maximum gray level and stretching all pixel values ​​to this range.

[0100] In the specific implementation process, the VGG19 pre-trained model is used to extract features from the pre-processed bridge images. The extracted features include the overall integrity of the bridge, signs of deformation or damage, and vehicle distribution features, such as the number and distribution of vehicles on the bridge. The bridge structural features and vehicle distribution features are then combined to form high-level bridge features.

[0101] A major challenge in the bridge collapse and vehicle fall detection tasks of this invention is the lack of training data. Due to the low incidence of such extreme accidents, coupled with the difficulties in data collection and annotation, the amount of available training data is extremely limited. In such small datasets, training a deep convolutional neural network directly from scratch can easily lead to overfitting and limited generalization performance. Therefore, this invention chooses to use the VGG19 pre-trained model as a feature extractor to alleviate this problem. The VGG19 pre-trained model was trained on the large ImageNet dataset and has already learned rich visual feature representations. Utilizing these pre-trained models to extract features allows us to directly enjoy the advantages gained from training on large datasets, without having to learn features from scratch. This not only accelerates the model's convergence speed but, more importantly, improves the model's generalization ability, effectively overcoming the overfitting risk associated with small datasets.

[0102] Specifically, the input preprocessed bridge image is passed through the convolutional and pooling layers of the VGG19 pre-trained model to extract high-level feature maps (high-level bridge features), which encode the local and global pattern information of the image.

[0103] In the specific implementation process, the neural network model includes a trained bridge detection sub-model and a vehicle detection sub-model;

[0104] The bridge detection sub-model predicts whether a bridge will collapse based on the extracted high-level bridge features, and outputs the corresponding bridge collapse prediction probability as the bridge collapse prediction result.

[0105] The vehicle detection sub-model performs vehicle target detection and vehicle fall classification based on the extracted high-level bridge features, and outputs the predicted vehicle bounding box and the probability of whether the vehicle has fallen as the prediction result of bridge fall.

[0106] Specifically:

[0107] 1) Preparation before training

[0108] To build a model capable of reliably detecting bridge collapses and vehicle falls, obtaining high-quality, diverse training data is crucial. Therefore, we will focus on data collection and annotation.

[0109] First, we need to collect a wide range of real bridge image data, covering various structural states such as intact, partially damaged, and severely collapsed. This data should also include extreme scenarios such as vehicles falling off the bridge, for example... Figure 2 , Figure 3 , Figure 4 and Figure 5 As shown, such a dataset can provide sufficient support for positive and negative examples for the model.

[0110] After obtaining the initial dataset, the images first need to be finely annotated. The annotation work mainly consists of two levels: first, binary labeling the integrity of the overall bridge structure (intact or collapsed); second, segmenting each vehicle in the image, accurately drawing its bounding box, and marking its fallen or intact state. This high-quality annotation will provide valuable supervisory information for model training.

[0111] To enhance data diversity and improve the model's generalization ability across different scenarios, we plan to employ various data augmentation techniques, such as image rotation, flipping, cropping, and color transformation. These transformations will allow us to generate more diverse training samples from the limited original data, enhancing the model's adaptability to factors such as lighting, viewpoint, and structural details.

[0112] Finally, following statistical principles, we will divide all the data into three independent subsets: a training set, a validation set, and a test set. The training set will be used for learning the model parameters; the validation set will guide model selection and hyperparameter tuning; and the test set will be strictly reserved and used only in the final stage of model development to evaluate its overall performance, ensuring fair and unbiased test results.

[0113] 2) Combining Figure 6 As shown, the training steps for the bridge detection sub-model are as follows:

[0114] S401: Obtain the VGG19 pre-trained model pre-trained on the ImageNet dataset;

[0115] S402: Standardize the training data (bridge image data) according to the format of the ImageNet dataset;

[0116] In this embodiment, the training data is standardized using operations such as subtracting the mean and dividing by the standard deviation to meet the input requirements of the pre-trained model. Since this invention selects a pre-trained model on the ImageNet dataset, the images need to be standardized according to the ImageNet dataset format. The specific formula is as follows:

[0117] X normalized =(X / 255-mean) imagenet ) / std imagenet ;

[0118] Where: mean imagenet std imagenet Here are the mean and standard deviation of the ImageNet dataset.

[0119] To improve data diversity, this embodiment can also apply data augmentation to generate more training samples. Geometric transformations include rotation, translation, scaling, flipping, cropping, and stretching, while color transformations include brightness adjustment, contrast adjustment, and color jitter.

[0120] Finally, the prepared training data (bridge image data) set is divided into a training set, a validation set, and a test set.

[0121] S403: Input the standardized training data into the VGG19 pre-trained model and extract the high-level features of the bridge through its convolutional and pooling layers;

[0122] S404: Input the extracted high-level bridge features into the bridge detection sub-model built on the fully connected layer, and output the corresponding bridge collapse prediction probability through the bridge detection sub-model;

[0123] In this embodiment, the bridge detection sub-model (fully connected layer) performs feature combination and transformation on the high-level features of the bridge, and finally outputs a binary prediction, which is classified as "intact" or "collapsed".

[0124] S405: Calculate the training loss of the bridge detection sub-model based on the actual values ​​of bridge collapse and the corresponding predicted probabilities of bridge collapse from the training data, and optimize the parameters of the bridge detection sub-model through backpropagation;

[0125] In this embodiment, Binary Cross Entropy is used as the loss function, and the weights of the bridge detection sub-model are optimized through backpropagation. The loss function formula for the bridge detection sub-model is as follows:

[0126]

[0127] BCE(p,y)=-[y×log(p)-(1-y)×log(1-p)];

[0128] In the formula: BCE(p,y) represents the degree of difference between the true value y and the predicted value p; Tal_BCE represents the average degree of difference between the true value and the predicted value of n training data; p i y represents the predicted value of the i-th training data; i This represents the true value of the i-th training data;

[0129] S406: Repeat steps S403 to S405 until the bridge detection sub-model converges.

[0130] 3) Combining Figure 7 As shown, the training steps for the vehicle detection sub-model are as follows:

[0131] S411: Obtain the VGG19 pre-trained model pre-trained on the ImageNet dataset;

[0132] S412: Obtain bridge images to be used as training data and standardize the training data according to the format of the ImageNet dataset;

[0133] S413: Input the standardized training data into the VGG19 pre-trained model and extract the high-level features of the bridge through its convolutional and pooling layers;

[0134] S414: Input the extracted high-level bridge features into the vehicle detection sub-model built based on the YOLOv1 target detection algorithm. The vehicle detection sub-model performs vehicle target detection, divides the image into multiple grid cells, and predicts the vehicle bounding box and the probability of whether the vehicle has fallen off the grid cell for each grid cell. Then, non-maximum suppression is used to filter and remove redundant vehicle bounding boxes. Finally, the predicted vehicle bounding box and the probability of whether the vehicle has fallen off the grid cell are output.

[0135] S415: Based on the true annotation of the vehicle bounding box and the true probability of whether the vehicle has fallen off the training data, the training loss and R2 value of the vehicle detection sub-model are calculated by combining the predicted vehicle bounding box and the class probability of whether the vehicle has fallen off the training data, and the parameters of the vehicle detection sub-model are optimized by backpropagation.

[0136] S416: Repeat steps S413 to S415 until the vehicle detection sub-model converges.

[0137] In practice, collapse early warning includes:

[0138] 1) Install audible and visual alarms around the bridge. When a risk of bridge collapse is detected, the audible and visual alarms will emit a loud sound and flashing lights to attract the attention of on-site personnel and management personnel.

[0139] 2) Install electronic display screens at major traffic arteries or prominent locations around the bridge to display the bridge's safety status in real time. When a risk of bridge collapse is detected, the display screen will show a warning message to remind passing vehicles and pedestrians to pay attention to safety.

[0140] 3) Utilize social media platforms or public broadcasting systems to issue early warnings about the risk of bridge collapse to the public, reminding them to avoid approaching or crossing the bridge to ensure personal safety;

[0141] 4) By adjusting the traffic signal control system, traffic flow and routes near the bridge can be changed to reduce traffic accidents that may be caused by the risk of bridge collapse;

[0142] 5) Establish a database of contacts (contacts may include managers and drivers). When a risk of bridge collapse is detected, send warning text messages or emails to the contacts to ensure that they are informed of the bridge's safety status in a timely manner and can take appropriate countermeasures.

[0143] In this embodiment, while issuing a collapse warning, the bridge image and the assessment results of the bridge collapse risk, the bridge already collapsed, or the vehicle falling are sent to the control center and the navigation system.

[0144] Specifically:

[0145] Two collapse thresholds were set when assessing the risk of bridge collapse: if the predicted probability of bridge collapse is less than the first collapse threshold, there is no risk of bridge collapse; if the predicted probability of bridge collapse is greater than or equal to the first collapse threshold and less than the second collapse threshold, it is determined that there is a risk of bridge collapse; if the predicted probability of bridge collapse is greater than or equal to the second collapse threshold, it is determined that there is a risk of bridge collapse.

[0146] A drop threshold is set when assessing the risk of a vehicle falling off: if the class probability of the vehicle bounding box is less than the drop threshold, there is no risk of the vehicle falling off; if the class probability of the vehicle bounding box is greater than or equal to the drop threshold, it is determined that there is a risk of the vehicle falling off.

[0147] The tiered early warning logic for bridge collapse includes:

[0148] 1) Level 3 Orange Alert: This indicates a risk of bridge collapse, such as significant downward deflection of the bridge deck, buckling with a concave-convex shape, or significant lateral misalignment and swaying deformation, but the bridge structure remains structurally intact. Sound and flashing lights are used to alert drivers and pedestrians to proceed with caution when entering the bridge; simultaneously, the cause of the bridge anomaly is sent to traffic police, control centers, and navigation systems.

[0149] 2) Level II Blue Alert: This alert indicates a risk of bridge collapse, such as structural fractures, significant changes in the structural system, road surface inability to maintain straightness, or irreversible swaying. Warnings are issued via audible and visual sirens and flashing lights; electronic displays show alerts to remind drivers and pedestrians to proceed with caution when entering the bridge; social media platforms and public address systems are used to disseminate the risk of bridge collapse to the public; traffic signal control systems are adjusted to modify traffic flow and routes near the bridge; traffic police are alerted, the control center is continuously notified, and the situation is immediately transmitted to navigation systems to avoid traffic congestion.

[0150] 3) Level 1 Red Alert: If both the risk of bridge collapse and the risk of vehicles falling are determined, sound and light alarms will be activated, and warning information will be displayed on electronic screens to remind drivers and pedestrians to be cautious when entering the bridge; warning information about the risk of bridge collapse will be released to the public through social media platforms or public broadcasting systems; traffic flow and driving routes near the bridge will be changed by adjusting traffic signal control systems; warning text messages or emails will be sent to contacts to ensure they are informed of the bridge's safety status in a timely manner; at the same time, the highest communication transmission frequency will be activated to continuously send on-site photos, monitoring videos, and real-time information to traffic police, control centers, and navigation systems.

[0151] Example 2:

[0152] This embodiment discloses a bridge collapse early warning system based on bridge images and neural networks, which is implemented based on the bridge collapse early warning method based on bridge images and neural networks in Embodiment 1.

[0153] like Figure 8 and Figure 9 As shown, the bridge collapse early warning system based on bridge images and neural networks includes:

[0154] Monitoring equipment is used to acquire images of bridges in locations prone to collapse.

[0155] In this embodiment, the installation locations of the monitoring equipment are selected based on the results of the selection of easily collapsible locations. These locations must ensure that the monitoring equipment can monitor the easily collapsible locations of the bridge without obstruction under favorable lighting conditions, so as to capture high-quality images and data in real time. In addition, the monitoring equipment should also cover a range of at least 100 meters from the easily collapsible locations of the bridge, so that there is sufficient distance to warn drivers to slow down in the event of a bridge collapse, and to facilitate the determination of whether a vehicle has fallen off the bridge.

[0156] A custom comprehensive analysis module is used to first preprocess the bridge image to obtain a preprocessed bridge image; secondly, feature extraction is performed on the preprocessed bridge image to obtain high-level bridge features including bridge structure and vehicle distribution; then, the high-level bridge features including bridge structure and vehicle distribution are input into a trained neural network model to output prediction results of bridge collapse and vehicle fall; finally, based on the prediction results of bridge collapse, the module assesses whether there is a risk of bridge collapse or vehicle fall.

[0157] In this embodiment, the installation location of the custom integrated analysis module was selected based on the principles of convenient power supply, easy maintenance, and no impact on the use of other public facilities. This ensures that the analysis module can operate under optimal conditions to support the system's efficient data processing and analysis.

[0158] The information storage module is used to store bridge images, as well as assessment results of the risk of bridge collapse, or whether the bridge has already collapsed or a vehicle has fallen off.

[0159] The early warning device is used to issue corresponding levels of collapse warnings based on the assessment results of bridge collapse risk and vehicle fall risk.

[0160] In this embodiment, the installation location of the early warning device is selected based on information about bridge vulnerabilities and the layout of monitoring equipment. Specifically, for lanes traveling in the same direction, the early warning device should be installed at least 20 meters before the bridge's vulnerable point to form an early warning network covering a range of 100 to 200 meters after the first installation point. The selection of the installation range also needs to comprehensively consider the collapse probability rating and local traffic flow information to ensure that areas with high traffic volume and complex road conditions receive an appropriate expansion of the early warning range, while areas with low traffic volume and clear road conditions have a more appropriate reduction in the early warning range.

[0161] Ultimately, monitoring equipment, customized integrated analysis modules, and early warning devices are connected into a unified information processing and transmission network via wired or wireless networks. This is key to achieving real-time monitoring, analysis, and early warning, ensuring efficient and accurate information flow.

[0162] Simultaneously, the instant messaging module immediately sends an alarm to traffic police, transmitting key data such as images and vehicle location information to the control center and navigation system. This ensures that road management personnel, police officers, emergency service personnel, and drivers can quickly and accurately understand the situation on-site, adjust their routes in a timely manner, or take other necessary measures to minimize the possibility of further accidents.

[0163] This invention acquires bridge images of locations prone to collapse and preprocesses these images. First, the invention pre-determines the locations of bridges prone to collapse and acquires targeted bridge images of these locations to enable subsequent bridge collapse prediction. This ensures the accuracy of bridge collapse prediction while reducing the amount of data processing and improving prediction efficiency. Second, preprocessing removes noise, blur, and unnecessary details from the bridge images, making the bridge structure and vehicle distribution clearer, thereby improving the accuracy of subsequent feature extraction. Furthermore, preprocessing ensures that bridge images from different sources, at different times, and under different conditions have a consistent format and quality, facilitating subsequent feature extraction and model training. Finally, preprocessing removes redundant information, reduces image size, lowers the computational complexity of subsequent processing, and ultimately improves the efficiency of bridge collapse prediction and processing.

[0164] This invention extracts features from preprocessed bridge images to obtain high-level bridge features that include bridge structure and vehicle distribution. Based on these extracted high-level features, it predicts bridge collapse and vehicle falls. First, the extracted high-level features, containing bridge structure and vehicle distribution, effectively describe the structural characteristics of the bridge and the distribution of vehicles, helping subsequent models accurately identify the state and potential risks of the bridge and vehicles on it. Second, by extracting high-level features, the high-dimensional data of the original bridge image can be converted into low-dimensional feature vectors, facilitating subsequent model processing and training. Finally, the high-level features extracted in this invention have stronger robustness and representativeness, enabling the model to better adapt to bridge images under different scenarios and conditions.

[0165] This invention, based on the extraction of high-level bridge features including bridge structure and vehicle distribution, further utilizes a neural network model to predict bridge collapse and vehicle falls. First, the neural network model can automatically learn and extract complex features from bridge images and optimize its prediction performance through training on a large amount of data, thus achieving efficient and accurate predictions of bridge collapse and vehicle falls. Second, the neural network model can handle complex nonlinear relationships, making it suitable for complex prediction tasks such as bridge collapse and vehicle falls, which are influenced by multiple factors, thus ensuring the effectiveness of bridge collapse prediction. Finally, the neural network model can self-adjust and optimize based on new data and circumstances, continuously improving its prediction performance and further enhancing the accuracy of bridge collapse prediction.

[0166] This invention assesses the risk of bridge collapse or vehicle fall based on bridge collapse predictions and issues corresponding levels of early warning. First, by assessing the risks of bridge collapse and vehicle fall, and establishing corresponding warning levels, real-time early warning of bridge collapse is achieved, enhancing emergency response capabilities. It can immediately alert surrounding vehicles and pedestrians upon detecting a collapse risk, providing a crucial time window to prevent secondary falls or reduce damage from accidents, thus effectively mitigating the safety risks posed by bridge collapse and vehicle fall. Second, issuing different levels of warning signals based on different risk levels and actual conditions helps management personnel and those on the bridge determine the bridge's collapse status based on the warning information, allowing for appropriate countermeasures and improving the effectiveness of bridge collapse early warning.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A bridge collapse early warning method based on bridge images and neural networks, characterized in that, include: S1: Determine the vulnerable locations of the bridge and obtain images of the bridge at these vulnerable locations; S2: Preprocess the bridge image to obtain a preprocessed bridge image; S3: Extract features from the preprocessed bridge image to obtain high-level bridge features that include bridge structure and vehicle distribution; In step S3, the pre-processed bridge image is used to extract features through the VGG19 pre-trained model. The bridge structural features, including the overall integrity of the bridge, signs of deformation or damage, and vehicle distribution features, including the number and distribution of vehicles on the bridge, are extracted. The bridge structural features and vehicle distribution features are then combined into high-level bridge features. S4: Input the high-level features of the bridge, including the bridge structure and vehicle distribution, into the trained neural network model, and output the prediction results of bridge collapse and vehicle fall. In step S4, the neural network model includes a trained bridge detection sub-model and a vehicle detection sub-model; The bridge detection sub-model predicts whether a bridge will collapse based on the extracted high-level bridge features, and outputs the corresponding bridge collapse prediction probability as the bridge collapse prediction result. The vehicle detection sub-model performs vehicle target detection and vehicle fall classification based on the extracted high-level bridge features, and outputs the predicted vehicle bounding box and the probability of vehicle fall as the prediction result of bridge fall. The training steps for the vehicle detection sub-model are as follows: S411: Obtain the VGG19 pre-trained model pre-trained on the ImageNet dataset; S412: Obtain bridge images to be used as training data and standardize the training data according to the format of the ImageNet dataset; S413: Input the standardized training data into the VGG19 pre-trained model and extract the high-level features of the bridge through its convolutional and pooling layers; S414: Input the extracted high-level bridge features into the vehicle detection sub-model built based on the YOLOv1 target detection algorithm. The vehicle detection sub-model performs vehicle target detection, divides the image into multiple grid cells, and predicts the vehicle bounding box and the probability of whether the vehicle has fallen off the grid cell for each grid cell. Then, non-maximum suppression is used to filter and remove redundant vehicle bounding boxes. Finally, the predicted vehicle bounding box and the probability of whether the vehicle has fallen off the grid cell are output. S415: Based on the true annotation of the vehicle bounding box and the true probability of whether the vehicle has fallen off the training data, the training loss and R2 value of the vehicle detection sub-model are calculated by combining the predicted vehicle bounding box and the class probability of whether the vehicle has fallen off the training data, and the parameters of the vehicle detection sub-model are optimized by backpropagation. S416: Repeat steps S413 to S415 until the vehicle detection sub-model converges; S5: Based on the prediction results of bridge collapse and vehicle fall, assess whether there is a risk of bridge collapse and vehicle fall, and issue an early warning of bridge collapse of corresponding level according to the assessment results.

2. The bridge collapse early warning method based on bridge images and neural networks as described in claim 1, characterized in that: In step S1, the bridge is subjected to structural mechanics analysis, vehicle and ship collision location analysis, and natural disaster vulnerability location analysis using finite element technology to identify potential weak points and collapse probability of the bridge, and to determine the locations of the bridge prone to collapse.

3. The bridge collapse early warning method based on bridge images and neural networks as described in claim 1, characterized in that: In step S2, the preprocessing of the bridge image includes image sharpening, noise removal, and contrast adjustment.

4. The bridge collapse early warning method based on bridge images and neural networks as described in claim 1, characterized in that: The training steps for the bridge detection sub-model are as follows: S401: Obtain the VGG19 pre-trained model pre-trained on the ImageNet dataset; S402: Obtain bridge images to be used as training data and standardize the training data according to the format of the ImageNet dataset; S403: Input the standardized training data into the VGG19 pre-trained model and extract the high-level features of the bridge through its convolutional and pooling layers; S404: Input the extracted high-level bridge features into the bridge detection sub-model constructed based on a fully connected layer and a sigmoid activation function, and output the corresponding bridge collapse prediction probability through the bridge detection sub-model; S405: Calculate the training loss of the bridge detection sub-model based on the actual values ​​of bridge collapse and the corresponding predicted probabilities of bridge collapse from the training data, and optimize the parameters of the bridge detection sub-model through backpropagation; The loss function formula for the bridge detection sub-model is as follows: ； ； In the formula: Represents the true value Compared with the predicted value The degree of difference between them; express The average degree of difference between the true and predicted values ​​of each training data point; Indicates the first Predicted values ​​for each training data set; Indicates the first The true value of each training data point; S406: Repeat steps S403 to S405 until the bridge detection sub-model converges.

5. The bridge collapse early warning method based on bridge images and neural networks as described in claim 1, characterized in that: In step S5, the collapse warning includes: 1) A sound and flashing light alarm will be activated to alert the user; 2) Display warning information on electronic screens; 3) Issue early warning information through social media platforms or public broadcasting systems; 4) Adjust traffic flow and routes near the bridge by modifying the traffic signal control system; 5) Establish a contact database in advance and send warning text messages or emails to the contacts.

6. The bridge collapse early warning method based on bridge images and neural networks as described in claim 1, characterized in that: In step S5, two collapse thresholds are set when assessing the risk of bridge collapse: if the predicted probability of bridge collapse is less than the first collapse threshold, there is no risk of bridge collapse; if the predicted probability of bridge collapse is greater than or equal to the first collapse threshold and less than the second collapse threshold, it is determined that there is a risk of bridge collapse; if the predicted probability of bridge collapse is greater than or equal to the second collapse threshold, it is determined that there is a risk of bridge collapse. A drop threshold is set when assessing the risk of a vehicle falling: if the probability of a vehicle falling is less than the drop threshold, then there is no risk of the vehicle falling. If the probability of a vehicle falling is greater than or equal to the falling threshold, then it is determined that there is a risk of the vehicle falling. The tiered early warning logic for bridge collapse includes: 1) Level 3 Orange Alert: It is determined that there is a risk of bridge collapse; sound and light alarms are issued and warning information is displayed on electronic screens; 2) Level II Blue Alert: This alert indicates a risk of bridge collapse; it uses audible and visual sirens and flashing lights to alert the system, displays warning information on electronic screens, disseminates warning information through social media platforms or public address systems, and adjusts traffic flow and routes near the bridge by modifying traffic signal control systems. 3) Level 1 Red Alert: If it is determined that there is a risk of bridge collapse and vehicle falling at the same time, sound and light alarms will be issued and warning information will be displayed on electronic screens; warning information will be issued through social media platforms or public broadcasting systems; traffic flow and driving routes near the bridge will be changed by adjusting traffic signal control systems; and warning text messages or emails will be sent to contacts.

7. A bridge collapse early warning system based on bridge images and neural networks, characterized in that: The implementation of the bridge collapse early warning method according to claim 1 specifically includes: Monitoring equipment is used to acquire images of bridges in locations prone to collapse. A custom comprehensive analysis module is used to first preprocess the bridge image to obtain a preprocessed bridge image; secondly, feature extraction is performed on the preprocessed bridge image to obtain high-level bridge features including bridge structure and vehicle distribution; then, the high-level bridge features including bridge structure and vehicle distribution are input into a trained neural network model to output prediction results of bridge collapse and vehicle fall; finally, based on the prediction results of bridge collapse, the module assesses whether there is a risk of bridge collapse or vehicle fall. The early warning device is used to issue corresponding levels of collapse warnings based on the assessment results of bridge collapse risk and vehicle fall risk.

Images