Computer vision-based bridge health monitoring anomaly identification system and method

Abstract

The present application relates to a bridge health monitoring anomaly identification system and method based on computer vision, the image recognition proposed by the present application relates to the use field of pattern recognition or machine learning based on video object classification. The present application comprises: an object construction mechanism for constructing a health monitoring anomaly identification model corresponding to the bridge to be monitored; a video processing mechanism for extracting three-machine position visual sorting data corresponding to the current monitoring time segment; a state recognition mechanism for intelligently identifying various structural abnormalities and their predicted occurrence time of the bridge to be monitored in the current monitoring time segment. The present application aims at the technical problem that the future bridge structural abnormal type and its occurrence time cannot be identified and warned synchronously, by using the health monitoring anomaly identification model designed for the bridge to be monitored, based on the multi-source basic data selected in a targeted manner, the synchronous intelligent identification of the future bridge structural abnormal type and its occurrence time is completed, thereby solving the above technical problems.

Description

Technical Field

[0001] The image recognition proposed in this invention relates to the fields of pattern recognition or machine learning using video object classification, and particularly to a bridge health monitoring anomaly identification system and method based on computer vision. Background Technology

[0002] Image recognition, pattern recognition, and machine learning are two major branches of artificial intelligence development. Combining them can further enhance the level of intelligence and reduce manual operations. For example, image recognition results can be used as neural network models employing pattern recognition or machine learning to obtain the status of various objects at the image acquisition site. Bridge health monitoring is a major application area, where computer vision data obtained through image recognition can be used as input to neural network models using pattern recognition or machine learning to achieve intelligent monitoring of bridge health status.

[0003] For example, Chinese invention patent publication CN119888496A proposes a bridge health status detection and early warning method and system based on image recognition, relating to the field of bridge health monitoring. The method includes: acquiring bridge crack images; preprocessing the bridge crack images to form a bridge crack image dataset; manually augmenting the bridge crack image dataset using a sliding window algorithm and extracting features to generate a feature dataset; training a model using a convolutional neural network to obtain a crack recognition model; collecting bridge health status data in real time using sensors and using the crack recognition model to detect cracks; modeling the bridge health status using time series analysis to generate dynamic damage prediction results; and implementing bridge health status assessment to provide early warning of bridge damage. This invention improves the accuracy and robustness of crack recognition by using an adaptive sliding window algorithm for bridge crack image augmentation and feature extraction, combined with a convolutional neural network and dynamic attention mechanism.

[0004] For example, Chinese invention patent publication CN119785128A proposes a method for anomaly identification in the health monitoring of long-span bridge structures based on image classification. This invention relates to the field of bridge structural health monitoring technology. The steps are as follows: Step 1: Generate anomaly samples using a generative adversarial network (GAN), where the anomaly samples include cracks, spalling, and corrosion. This image classification-based method for anomaly identification in the health monitoring of long-span bridge structures alleviates data scarcity by generating and optimizing anomaly samples through a generative adversarial network; the dynamic generation-learning-optimization closed-loop system and distributed collaborative platform reduce annotation costs; weakly supervised learning, knowledge graphs, and automated quality inspection tools improve the stability of annotation quality; and self-supervised learning technology improves the utilization rate of unlabeled data during multimodal data fusion, comprehensively solving the problems of data annotation and quality, and providing strong support for bridge structural health monitoring.

[0005] Therefore, it is evident that in existing technologies, when combining image recognition, pattern recognition, and machine learning—two major branches of artificial intelligence—for intelligent monitoring of bridge health, it is possible to identify real-time anomalies and provide early warnings of bridge damage. However, the early warnings provided by existing technologies do not pinpoint the exact time when a certain type of anomaly will occur in the bridge. Instead, they use real-time damage status to warn of potential further damage. Furthermore, they cannot predict the exact time when a certain type of anomaly will occur in advance. This results in a lack of foresight in anomaly identification by bridge health monitoring departments, making it impossible to address various structural anomalies in advance and leading to poor efficiency and effectiveness in bridge health maintenance. Summary of the Invention

[0006] To address the technical problems in existing technologies, this invention provides a bridge health monitoring anomaly identification system and method based on computer vision. This system employs a health monitoring anomaly identification model customized for the bridge to be monitored. Based on selectively chosen multi-source basic data, it synchronously and intelligently identifies various structural anomalies that will occur on the same bridge within the current monitoring time segment (which serves as a future monitoring time segment), as well as the predicted occurrence times of these anomalies. The synchronous intelligent identification results are then used for various early warning operations, including on-site early warning information display, real-time wireless transmission of early warning information, and on-site early warning information broadcasting. This provides a targeted and reliable early warning mechanism for bridge structural anomaly monitoring, improving the efficiency and effectiveness of bridge health maintenance.

[0007] According to one aspect of the present invention, a computer vision-based bridge health monitoring anomaly identification system is provided, the system comprising:

[0008] An object construction mechanism is used to use a convolutional neural network that has been trained multiple times as a health monitoring anomaly identification model for the bridge to be monitored.

[0009] The directional acquisition mechanism is used to collect video from the top, left, and right sides of the environment of the bridge under monitoring within the current monitoring time segment by using three drone aerial cameras located at three shooting positions directly above, to the left, and to the right of the bridge under monitoring. The distance from the three shooting positions to the bridge under monitoring is a set shooting distance.

[0010] The video processing mechanism, connected to the directional acquisition mechanism, is used to extract the sorting data of each frame of the video directly above the environment of the bridge to be monitored, the sorting data of each frame of the video directly to the left, and the sorting data of each frame of the video directly to the right within the current monitoring time segment, so as to serve as the three-camera visual sorting data corresponding to the current monitoring time segment.

[0011] Information analysis agency, used to analyze each relevant captured information segment of the current monitoring time period;

[0012] The status recognition mechanism is connected to the object construction mechanism, video processing mechanism, and information analysis mechanism, respectively. It is used to identify various structural anomalies that will occur on the bridge under monitoring and the expected time of occurrence of various structural anomalies in the current monitoring time segment, based on the set shooting distance, the bridge type code value, the year of completion, the span length, the bridge design height and salt spray area marking, the duration of the current monitoring time segment, the three-camera visual sorting data corresponding to the current monitoring time segment, and the relevant captured information of the current monitoring time segment. The current monitoring time segment starts from the current time.

[0013] An anomaly warning mechanism, connected to a status recognition mechanism, is used to perform anomaly warning operations for the current monitoring time segment based on intelligent recognition results.

[0014] According to another aspect of the present invention, a method for identifying anomalies in bridge health monitoring based on computer vision is provided, the method comprising:

[0015] The convolutional neural network, trained multiple times, is used as the health monitoring anomaly identification model for the bridge to be monitored.

[0016] Three drones, positioned directly above, to the left, and to the right of the bridge to be monitored, respectively, captured video footage of the environment above, to the left, and to the right of the bridge within the current monitoring time segment. The distance from each of the three camera positions to the bridge was a set shooting distance.

[0017] Extract the sorting data of each frame of the video directly above the environment of the bridge to be monitored, the sorting data of each frame of the video directly to the left, and the sorting data of each frame of the video directly to the right within the current monitoring time segment, and use them as the three-camera visual sorting data corresponding to the current monitoring time segment.

[0018] Analyze the relevant capture information for each segment of the current monitoring time;

[0019] Using the health monitoring anomaly identification model corresponding to the bridge to be monitored, based on the set shooting distance, the bridge type code value, the year of completion, the span length, the bridge design height and salt spray area marking, the duration of the current monitoring time segment, the three-camera visual sorting data corresponding to the current monitoring time segment, and the relevant capture information of each part of the current monitoring time segment, the model can intelligently identify various structural anomalies that will occur on the bridge to be monitored within the current monitoring time segment and the expected time of occurrence of each type of structural anomaly. The current monitoring time segment starts at the current time.

[0020] Based on the intelligent recognition results, perform abnormal early warning operations for the current monitoring time segment.

[0021] Therefore, it can be seen that the present invention has at least the following key inventive points:

[0022] The first invention point: Using the same artificial intelligence model and based on multi-source basic data, it can synchronously and intelligently identify various structural anomalies that will occur on the same bridge within the current monitoring time segment, which is a future monitoring time segment, as well as the expected occurrence time of various structural anomalies. The synchronous intelligent identification results are then used for various early warning operations, including on-site early warning information display, real-time wireless transmission of early warning information, and on-site early warning information broadcasting. This provides a targeted and reliable early warning mechanism for bridge structural anomaly monitoring, improving the efficiency and effectiveness of bridge health maintenance.

[0023] The second invention point: In order to achieve synchronous intelligent identification of various structural anomalies that will occur within the current monitoring time segment and the expected occurrence time of various structural anomalies, a health monitoring anomaly identification model with a customized structure design for the bridge to be monitored is adopted. In this model, a convolutional neural network that has been trained multiple times is used as the health monitoring anomaly identification model for the bridge to be monitored. The convolutional neural network includes a single input layer, multiple convolutional layers, a single activation function layer, a single pooling layer, and a single fully connected layer connected in sequence. The number of convolutional layers has the same numerical trend as the span length of the bridge to be monitored, and the number of training times is proportional to the completion year of the bridge to be monitored. The customized structure design of the above-mentioned health monitoring anomaly identification model increases the monitoring accuracy of future structural anomaly types and expected occurrence times.

[0024] The third invention point: In each training iteration of the convolutional neural network, various structural anomalies occurring on the bridge under monitoring within a certain past monitoring time segment, along with the times of occurrence of these anomalies, are used as the output content of the convolutional neural network. The set shooting distance, bridge type code value, completion year, span length, bridge design height, salt spray area marker, occupation duration of the aforementioned past monitoring time segment, the three-camera visual sorting data corresponding to the aforementioned past monitoring time segment, and the relevant capture information of the current monitoring time segment are used as the input content of the convolutional neural network to complete this training, thereby ensuring the training effect of the convolutional neural network in each training iteration.

[0025] The fourth invention point: In order to achieve synchronous intelligent identification of various structural anomalies that will occur within the current monitoring time segment and the expected occurrence time of various structural anomalies, comprehensive and sufficient multi-source basic data were selected, including the set shooting distance, bridge type code value of the bridge to be monitored, completion year, span length, bridge design height and salt spray area marking, occupation duration of the current monitoring time segment, three-camera visual sorting data corresponding to the current monitoring time segment, and each relevant capture information of the current monitoring time segment. The comprehensive and sufficient selection of the above multi-source basic data further increases the monitoring accuracy of future structural anomaly types and expected occurrence times.

[0026] The fifth point of invention: Specifically, the three-camera visual sorting data corresponding to the current monitoring time segment includes the sorting data of each frame of the video directly above, to the left of, and to the right of the environment where the bridge to be monitored is located within the current monitoring time segment. The sorting data of each frame of each video is the depth value, gray value, vertical coordinate value, and horizontal coordinate value of each pixel in the area occupied by the bridge to be monitored in that frame, as well as the gray value gradient value, vertical coordinate value, and horizontal coordinate value of each edge pixel in the area occupied by the bridge to be monitored in that frame. The relevant captured information of the current monitoring time segment includes the average cable tension, average ambient temperature, average ambient humidity, average on-site wind speed, on-site wind direction, total on-site traffic volume, and average load weight of the bridge to be monitored within the current monitoring time segment. Thus, a customized data structure design has been completed for the multi-source basic data used for synchronous intelligent identification. Attached Figure Description

[0027] The embodiments of the present invention will now be described with reference to the accompanying drawings, wherein:

[0028] Figure 1 This is a schematic diagram of the technical process of the bridge health monitoring anomaly identification system and method based on computer vision according to the present invention.

[0029] Figure 2 This is an internal structure diagram of a computer vision-based bridge health monitoring anomaly identification system according to a first embodiment of the present invention.

[0030] Figure 3 This is an internal structure diagram of a computer vision-based bridge health monitoring anomaly identification system according to a second embodiment of the present invention.

[0031] Figure 4 This is an internal structure diagram of a computer vision-based bridge health monitoring anomaly identification system according to a third embodiment of the present invention.

[0032] Figure 5 This is an internal structure diagram of a computer vision-based bridge health monitoring anomaly identification system according to a fourth embodiment of the present invention.

[0033] Figure 6 This is an internal structure diagram of a computer vision-based bridge health monitoring anomaly identification system according to the fifth embodiment of the present invention.

[0034] Figure 7 The above is a flowchart illustrating the steps of a computer vision-based bridge health monitoring anomaly identification method according to the sixth embodiment of the present invention. Detailed Implementation

[0035] like Figure 1 The diagram illustrates the technical flow of a bridge health monitoring anomaly identification system and method based on computer vision according to the present invention. The image recognition proposed in this invention relates to the application of pattern recognition or machine learning based on video object classification.

[0036] The specific technical process of this invention is as follows:

[0037] Technical Process A: Customize a health monitoring anomaly identification model for the bridge to be monitored, such as... Figure 1 As shown, this is used to achieve synchronous intelligent identification of various structural anomalies that will occur within the current monitoring time segment and the expected occurrence time of each structural anomaly.

[0038] Specifically, the current monitoring time segment starts at the current moment. Therefore, the current monitoring time segment is a future time segment. The bridge structure anomaly types and the expected time of anomaly occurrence identified by subsequent synchronous intelligent identification are all future data.

[0039] Specifically, the customized structure of the health monitoring anomaly identification model for the bridge to be monitored is mainly reflected in the following aspects:

[0040] Aspect 1: The health monitoring anomaly identification model for the bridge under monitoring is a convolutional neural network that has been trained multiple times, and the number of training times is proportional to the completion year of the bridge under monitoring.

[0041] For example, if the bridge to be monitored is 6 years old, the number of training sessions is 600; if the bridge to be monitored is 7 years old, the number of training sessions is 700; if the bridge to be monitored is 8 years old, the number of training sessions is 800; if the bridge to be monitored is 9 years old, the number of training sessions is 900, and so on.

[0042] Aspect 2: The convolutional neural network used consists of a single input layer, multiple convolutional layers, a single activation function layer, a single pooling layer, and a single fully connected layer connected in sequence;

[0043] Thirdly: In the convolutional neural network used, the number of convolutional layers shows the same numerical trend as the span length of the bridge to be monitored;

[0044] For example, the span of the bridge to be monitored is 200 meters and the number of convolutional layers is 2; the span of the bridge to be monitored is 500 meters and the number of convolutional layers is 5; the span of the bridge to be monitored is 1000 meters and the number of convolutional layers is 10; the span of the bridge to be monitored is 1500 meters and the number of convolutional layers is 15, and so on.

[0045] Fourthly, in each training iteration of the convolutional neural network, various structural anomalies occurring on the bridge under monitoring within a specific past monitoring time segment, along with the times of occurrence of these anomalies, are used as the output of the convolutional neural network. The set shooting distance, bridge type code value, completion year, span length, bridge design height, salt spray area marker, occupation duration of the specified past monitoring time segment, the three-camera visual sorting data corresponding to the specified past monitoring time segment, and relevant capture information from the current monitoring time segment are used as the input of the convolutional neural network. This completes the training and ensures the effectiveness of each training iteration of the convolutional neural network.

[0046] In this way, the stability and reliability of the synchronous intelligent identification results of the bridge structure anomaly types and the expected time of anomaly occurrence are ensured through the customized structural designs mentioned above.

[0047] Technical Process B: To achieve synchronous intelligent identification of various structural anomalies that will occur within the current monitoring time segment and the expected occurrence time of various structural anomalies, comprehensive and sufficient multi-source basic data were selected in a targeted manner;

[0048] Specifically, the multi-source basic data includes the set shooting distance, the bridge type code value of the bridge to be monitored, the year of completion, the span length, the design height of the bridge and the salt spray area marker, the occupation duration of the current monitoring time segment, the three-camera visual sorting data corresponding to the current monitoring time segment, and the relevant capture information of each part of the current monitoring time segment. The comprehensive and sufficient selection of the above multi-source basic data further increases the monitoring accuracy of future structural anomaly types and the expected time of occurrence.

[0049] exist Figure 1 In the middle, the auxiliary data includes the set shooting distance, the bridge type code value of the bridge to be monitored, the year of completion, the span length, the design height of the bridge and the salt spray area mark, and the duration of the current monitoring time segment;

[0050] More specifically, the three-camera visual sorting data corresponding to the current monitoring time segment includes the sorting data of each frame of the video directly above the environment where the bridge to be monitored is located within the current monitoring time segment, the sorting data of each frame of the video directly to the left, and the sorting data of each frame of the video directly to the right. The sorting data of each frame of each video is the depth value, gray value, vertical coordinate value, and horizontal coordinate value of each pixel in the area of ​​the image occupied by the bridge to be monitored in that frame, as well as the gray value gradient value, vertical coordinate value, and horizontal coordinate value of each edge pixel in the area of ​​the image occupied by the bridge to be monitored in that frame.

[0051] More specifically, the relevant captured information for each monitoring time segment includes the average cable tension, average ambient temperature, average ambient humidity, average on-site wind speed, on-site wind direction indication, total on-site traffic volume, and average load weight of the bridge under monitoring within the current monitoring time segment.

[0052] In this way, a customized data structure design was completed for the multi-source basic data used for synchronous intelligent identification;

[0053] Furthermore, through the targeted selection of comprehensive and sufficient multi-source basic data, the stability and reliability of the synchronous intelligent identification results of bridge structural anomaly types and the expected occurrence time of anomalies are further guaranteed.

[0054] Technical Process C: Utilizing the health monitoring anomaly identification model customized for the bridge under monitoring using Technical Process A, and based on multi-source basic data specifically selected using Technical Process B, synchronous intelligent identification of various structural anomalies that will occur on the bridge under monitoring within the current monitoring time segment, as well as the predicted occurrence times of these anomalies, is completed. Figure 1 As shown;

[0055] Specifically, the structural anomalies obtained by synchronous intelligent identification may be of one type or multiple types. They may also be that there are no structural anomalies in the current monitoring time segment. The corresponding structural anomaly type can be determined by whether the expected occurrence time corresponding to the structural anomaly type is an empty character.

[0056] Technical Process D: Based on the synchronous intelligent identification results of Technical Process C, execute anomaly warning operations for the current monitoring time segment;

[0057] Specifically, various structural anomalies that will occur on the bridge under monitoring within the current monitoring time segment, as well as the expected time of occurrence of various structural anomalies, are displayed on-site as anomaly warning information, or wirelessly transmitted to a remote bridge monitoring server, or broadcast on-site.

[0058] This provides a targeted and reliable early warning mechanism for monitoring structural anomalies in bridges, improving the efficiency and effectiveness of bridge health maintenance.

[0059] Therefore, it can be seen that, through the coordinated operation of the above-mentioned technical processes, the present invention can use the same artificial intelligence model, based on multi-source basic data, to complete the synchronous intelligent identification of various structural anomalies that will occur on the same bridge within the current monitoring time segment, which is a future monitoring time segment, as well as the expected occurrence time of various structural anomalies. The synchronous intelligent identification results are then used for various early warning operations, including on-site early warning information display, real-time wireless transmission of early warning information, and on-site early warning information broadcasting. This provides a targeted and reliable early warning mechanism for bridge structural anomaly monitoring, improving the efficiency and effectiveness of bridge health maintenance.

[0060] The key points of this invention are: synchronous intelligent identification of various structural anomalies that will occur in the bridge within future monitoring time segments and the expected occurrence time of various structural anomalies; health monitoring anomaly identification model for different customized structural designs of different bridges; targeted selection of comprehensive and sufficient multi-source basic data; and targeted design of each training of the convolutional neural network.

[0061] The present invention will now be described in detail by way of embodiments of the computer vision-based bridge health monitoring anomaly identification system and method.

[0062] First Embodiment

[0063] Figure 2 This is an internal structure diagram of a computer vision-based bridge health monitoring anomaly identification system according to a first embodiment of the present invention.

[0064] like Figure 2 As shown, the computer vision-based bridge health monitoring anomaly identification system includes the following components:

[0065] An object construction mechanism is used to use a convolutional neural network that has been trained multiple times as a health monitoring anomaly identification model for the bridge to be monitored.

[0066] Specifically, in this invention, different health monitoring anomaly identification models are designed for different bridges, and the health monitoring anomaly identification models are designed in various customized ways to achieve different structures for the health monitoring anomaly identification models of different bridges;

[0067] The directional acquisition mechanism is used to collect video from the top, left, and right sides of the environment of the bridge under monitoring within the current monitoring time segment by using three drone aerial cameras located at three shooting positions directly above, to the left, and to the right of the bridge under monitoring. The distance from the three shooting positions to the bridge under monitoring is a set shooting distance.

[0068] Specifically, the three camera positions completed the all-round visual data collection of the bridge to be monitored, thus providing key data for the subsequent status identification of the bridge to be monitored;

[0069] The video processing mechanism, connected to the directional acquisition mechanism, is used to extract the sorting data of each frame of the video directly above the environment of the bridge to be monitored, the sorting data of each frame of the video directly to the left, and the sorting data of each frame of the video directly to the right within the current monitoring time segment, so as to serve as the three-camera visual sorting data corresponding to the current monitoring time segment.

[0070] Obviously, it is impractical to use every frame of the video from the top, left, and right sides of the environment of the bridge under monitoring within the current monitoring time segment as input for the subsequent model. This is because the amount of data in each frame is too large, and video processing is required to sort out the key visual data so that the less data-intensive three-camera visual sorting data can be used as input for the subsequent model.

[0071] Information analysis agency, used to analyze each relevant captured information segment of the current monitoring time period;

[0072] Specifically, the current monitoring time segment starts at the current moment. Therefore, the current monitoring time segment is a future time segment. The bridge structure anomaly types and the expected time of anomaly occurrence identified by subsequent synchronous intelligent identification are all future data.

[0073] The status recognition mechanism is connected to the object construction mechanism, video processing mechanism, and information analysis mechanism, respectively. It is used to identify various structural anomalies that will occur on the bridge under monitoring and the expected time of occurrence of various structural anomalies in the current monitoring time segment, based on the set shooting distance, the bridge type code value, the year of completion, the span length, the bridge design height and salt spray area marking, the duration of the current monitoring time segment, the three-camera visual sorting data corresponding to the current monitoring time segment, and the relevant captured information of the current monitoring time segment. The current monitoring time segment starts from the current time.

[0074] In this way, the synchronous intelligent identification of the future abnormal state types of the same bridge and the time of their occurrence is completed;

[0075] An anomaly warning mechanism, connected to a status recognition mechanism, is used to perform anomaly warning operations for the current monitoring time segment based on intelligent recognition results;

[0076] Specifically, abnormal warnings can be displayed on-site, transmitted wirelessly to a remote bridge monitoring server, or broadcast on-site, among other methods.

[0077] Among them, the analysis of each relevant captured information in the current monitoring time segment includes: the average cable tension, average ambient temperature, average ambient humidity, average on-site wind speed, on-site wind direction, total on-site traffic volume, and average load weight of the bridge under monitoring in the previous time segment before the current monitoring time segment as each relevant captured information in the current monitoring time segment.

[0078] For example, if the current time is 8:00 AM, the current monitoring time segment is from 8:00 AM to 8:10 AM, and the previous time segment was from 7:50 AM to 8:00 AM, then the relevant captured information for each segment of the current monitoring time segment is: the average cable tension, average ambient temperature, average ambient humidity, average on-site wind speed, on-site wind direction, total on-site traffic volume, and average load weight of the bridge under monitoring from 7:50 AM to 8:00 AM.

[0079] Among them, the abnormal early warning operation based on the intelligent recognition results for the current monitoring time segment includes: displaying on-site various structural abnormalities that will occur on the bridge to be monitored within the current monitoring time segment, as well as the expected occurrence time of various structural abnormalities;

[0080] For example, an on-site LED display array or an on-site LCD display array can be selected to display various structural anomalies that will occur on the bridge under monitoring within the current monitoring time segment, as well as the expected time of occurrence of various structural anomalies, as anomaly warning information on-site;

[0081] Each type of structural anomaly is one of the following: excessive deflection of the main beam, excessive displacement of the pier, excessive settlement of the pier, transverse cracks in the bridge deck, longitudinal cracks in the bridge deck, efflorescence of the bridge deck concrete, corrosion of the reinforcing steel, cracking of the pier concrete, and tilting of the guardrail.

[0082] Specifically, different types of structural anomalies have different numerical type codes.

[0083] The data sorted for each frame of each video consists of the depth value, grayscale value, vertical coordinate value and horizontal coordinate value of each pixel in the area occupied by the bridge to be monitored in that frame, as well as the grayscale gradient value, vertical coordinate value and horizontal coordinate value of each edge pixel in the area occupied by the bridge to be monitored in that frame.

[0084] For example, in each frame of each video, the pixel at the bottom left of the frame is taken as the origin of the horizontal coordinate system, the leftmost column of pixels in the frame is taken as the positive vertical axis of the horizontal coordinate system, and the bottom row of pixels in the frame is taken as the positive horizontal axis of the horizontal coordinate system, thus completing the construction of the horizontal coordinate system corresponding to the frame.

[0085] Among them, different bridge types correspond to different bridge type code values, and different bridge types include cable-stayed bridges, suspension bridges, arch bridges, railway bridges and urban overpass bridges. The salt spray area identifier of the bridge to be monitored is used to indicate whether the bridge to be monitored is in a salt spray area, and the resolution of the three drone aerial cameras is the same.

[0086] For example, different bridge types correspond to different bridge type codes, and different bridge types include cable-stayed bridges, suspension bridges, arch bridges, railway bridges and urban overpass bridges. The salt spray area identifier of the bridge to be monitored is used to indicate whether the bridge to be monitored is in a salt spray area, and the resolution of the three drone aerial cameras is the same, including: the resolution of the three drone aerial cameras can all be 8K resolution.

[0087] Among them, using a convolutional neural network trained multiple times as the health monitoring anomaly identification model for the bridge to be monitored includes: using a convolutional neural network trained multiple times as the health monitoring anomaly identification model for the bridge to be monitored, the convolutional neural network includes a single input layer, multiple convolutional layers, a single activation function layer, a single pooling layer and a single fully connected layer connected in sequence, the number of convolutional layers has the same numerical trend as the span length of the bridge to be monitored, and the number of training times is proportional to the completion year of the bridge to be monitored;

[0088] Specifically, the numerical variation trend of the number of convolutional layers and the span length of the bridge to be monitored is the same, which means that after the curve of the span length of the bridge to be monitored and the curve of the number of convolutional layers are normalized, the two processed curves can completely overlap.

[0089] For example, the span of the bridge to be monitored is 200 meters and the number of convolutional layers is 2; the span of the bridge to be monitored is 500 meters and the number of convolutional layers is 5; the span of the bridge to be monitored is 1000 meters and the number of convolutional layers is 10; the span of the bridge to be monitored is 1500 meters and the number of convolutional layers is 15, and so on.

[0090] For example, if the bridge to be monitored is 6 years old, the number of training sessions is 600; if the bridge to be monitored is 7 years old, the number of training sessions is 700; if the bridge to be monitored is 8 years old, the number of training sessions is 800; if the bridge to be monitored is 9 years old, the number of training sessions is 900, and so on.

[0091] In each training iteration of the convolutional neural network, various structural anomalies occurring on the bridge under monitoring within a specific past monitoring time segment, along with the times of occurrence of these anomalies, are used as the output of the convolutional neural network. The set shooting distance, bridge type code value, completion year, span length, bridge design height, salt spray area marker, occupation duration of the specified past monitoring time segment, the three-camera visual sorting data corresponding to the specified past monitoring time segment, and relevant capture information from the current monitoring time segment are used as the input of the convolutional neural network to complete this training.

[0092] Second Embodiment

[0093] Figure 3 This is an internal structure diagram of a computer vision-based bridge health monitoring anomaly identification system according to a second embodiment of the present invention.

[0094] like Figure 3 As shown, compared to Figure 2 The computer vision-based bridge health monitoring anomaly identification system further includes:

[0095] The wireless transmission mechanism, connected to the status identification mechanism, is used to synchronously and wirelessly transmit the various structural anomalies that will occur on the bridge under monitoring within the current monitoring time segment, as well as the expected occurrence time of each structural anomaly, to the remote bridge monitoring server.

[0096] The method of synchronously and wirelessly transmitting various structural anomalies that will occur on the bridge under monitoring during the current monitoring time segment, as well as the expected occurrence time of each structural anomaly, to a remote bridge monitoring server includes: packaging various structural anomalies that will occur on the bridge under monitoring during the current monitoring time segment, as well as the expected occurrence time of each structural anomaly, into the same network data packet, and then wirelessly transmitting the network data packet to the remote bridge monitoring server.

[0097] For example, the process of packaging all types of structural anomalies that will occur on the bridge under monitoring during the current monitoring time segment, along with the expected occurrence times of each type of structural anomaly, into the same network data packet, and then wirelessly transmitting the network data packet to a remote bridge monitoring server, includes: the same network data packet being an IP data packet.

[0098] Third Embodiment

[0099] Figure 4 This is an internal structure diagram of a computer vision-based bridge health monitoring anomaly identification system according to a third embodiment of the present invention.

[0100] like Figure 4 As shown, compared to Figure 2 The computer vision-based bridge health monitoring anomaly identification system further includes:

[0101] The acoustic early warning mechanism, connected to the status recognition mechanism, is used to receive information on various structural anomalies that will occur on the bridge under monitoring within the current monitoring time segment, as well as the estimated time of occurrence of each structural anomaly.

[0102] The acoustic early warning mechanism is also used to broadcast on-site various structural anomalies that will occur on the bridge under monitoring during the current monitoring time segment, as well as the expected time of occurrence of various structural anomalies.

[0103] Specifically, the acoustic early warning mechanism is also used to broadcast on-site various structural anomalies that will occur on the bridge under monitoring within the current monitoring time segment, as well as the expected occurrence time of various structural anomalies. This includes: the acoustic early warning mechanism has a built-in voice broadcast chip, a microcontroller, and a serial communication interface.

[0104] Fourth embodiment

[0105] Figure 5 This is an internal structure diagram of a computer vision-based bridge health monitoring anomaly identification system according to a fourth embodiment of the present invention.

[0106] like Figure 5 As shown, compared to Figure 2 The computer vision-based bridge health monitoring anomaly identification system further includes:

[0107] The synchronous control mechanism, connected to the directional acquisition mechanism, is used to perform synchronous control of three drone aerial photography devices located at three camera positions directly above, to the left and to the right of the bridge to be monitored.

[0108] Specifically, a GAL device can be used to implement a synchronization control mechanism, which is used to perform synchronous control of three drone aerial photography devices located at three camera positions directly above, to the left and to the right of the bridge to be monitored.

[0109] The synchronous control of three drones positioned at three camera locations—directly above, to the left of, and to the right of the bridge to be monitored—includes controlling the three drones to simultaneously perform shooting actions to collect video footage from directly above, to the left of, and to the right of the environment surrounding the bridge to be monitored within the current monitoring time segment.

[0110] Fifth Embodiment

[0111] Figure 6 This is an internal structure diagram of a computer vision-based bridge health monitoring anomaly identification system according to the fifth embodiment of the present invention.

[0112] like Figure 6 As shown, compared to Figure 2 The computer vision-based bridge health monitoring anomaly identification system further includes:

[0113] The timing service provider is connected to the status identification provider and the directional data acquisition provider respectively, and is used to provide the timing services required by the status identification provider and the directional data acquisition provider respectively.

[0114] The timing service mechanism is connected to the status identification mechanism and the directional acquisition mechanism respectively, and is used to provide the status identification mechanism and the directional acquisition mechanism with their respective timing services. The timing service mechanism provides the status identification mechanism and the directional acquisition mechanism with their respective timing services by providing a reference clock signal with a square waveform of a set frequency.

[0115] Specifically, the timing service can incorporate a quartz oscillator unit to generate a reference clock signal with a square waveform at a set frequency.

[0116] Next, various embodiments of the present invention will be further described.

[0117] Optionally, within the above embodiments, in the computer vision-based bridge health monitoring anomaly identification system:

[0118] Based on the gray value distribution range corresponding to the bridge, the area of ​​the image occupied by the bridge to be monitored in each frame is identified. The gray value distribution range corresponding to the bridge is formed by limiting the upper limit gray value and the lower limit gray value corresponding to the bridge, and the upper limit gray value corresponding to the bridge is greater than the lower limit gray value corresponding to the bridge.

[0119] Specifically, the upper limit gray value and the lower limit gray value of the bridge are both between 0 and 255.

[0120] Specifically, the method involves identifying the area occupied by the bridge to be monitored in each frame based on the grayscale value distribution range corresponding to the bridge. The grayscale value distribution range corresponding to the bridge is a value distribution range formed by limiting the upper limit grayscale value and the lower limit grayscale value corresponding to the bridge, and the upper limit grayscale value corresponding to the bridge is greater than the lower limit grayscale value corresponding to the bridge. This includes: taking the pixels in each frame whose grayscale value is within the grayscale value distribution range corresponding to the bridge as the suspected bridge to be monitored pixels in that frame; removing isolated pixels from each of the suspected bridge to be monitored pixels in that frame to obtain the remaining suspected bridge to be monitored pixels; and fitting the remaining suspected bridge to be monitored pixels to obtain the area occupied by the bridge to be monitored in that frame.

[0121] And, optionally, within the above embodiments, in the computer vision-based bridge health monitoring anomaly identification system:

[0122] The average cable tension, average ambient temperature, average ambient humidity, average on-site wind speed, on-site wind direction, total on-site traffic volume, and average load weight of the bridge under monitoring in the previous time segment are used as the relevant captured information of the bridge under monitoring in the current monitoring time segment. This includes: the arithmetic mean of the remaining cable tensions obtained after removing the maximum and minimum values ​​of each cable tension corresponding to each time segment at even intervals in the current monitoring time segment; the average cable tension of the bridge under monitoring in the previous time segment; and the arithmetic mean of the remaining on-site wind speeds obtained after removing the maximum and minimum values ​​of each on-site wind speed corresponding to each time segment at even intervals in the current monitoring time segment.

[0123] Specifically, the following calculations can be performed using an MCU chip: the arithmetic mean of the remaining cable tensions obtained after removing the maximum and minimum values ​​of each cable tension at evenly spaced moments within the current monitoring time segment is used as the average cable tension of the bridge in the previous time segment before the current monitoring time segment; and the arithmetic mean of the remaining field wind speeds obtained after removing the maximum and minimum values ​​of each field wind speed at evenly spaced moments within the current monitoring time segment is used as the average field wind speed of the bridge in the current monitoring time segment.

[0124] The data captured for the bridge under monitoring in the current monitoring time segment includes the average cable tension, average ambient temperature, average ambient humidity, average on-site wind speed, on-site wind direction, total on-site traffic volume, and average load weight within the time segment preceding the current monitoring time segment. It also includes the arithmetic mean of the remaining ambient temperatures obtained after removing the maximum and minimum values ​​from the ambient temperatures at evenly spaced intervals within the current monitoring time segment, and the arithmetic mean of the remaining ambient humidity values ​​obtained after removing the maximum and minimum values ​​from the ambient humidity values ​​at evenly spaced intervals within the current monitoring time segment, which is used as the average ambient temperature of the bridge under monitoring in the current monitoring time segment.

[0125] In addition, the average cable tension, average ambient temperature, average ambient humidity, average on-site wind speed, on-site wind direction indicator, total on-site traffic volume, and average load weight of the bridge under monitoring in the previous time segment before the current monitoring time segment are also included as the relevant captured information of the bridge under monitoring in the current monitoring time segment. The arithmetic mean of the remaining multiple bridge load weights obtained after removing the maximum and minimum values ​​of each bridge load weight corresponding to each time interval at even intervals in the current monitoring time segment is used as the average total load of the bridge under monitoring in the current monitoring time segment, and different wind directions have different wind direction indicators.

[0126] Sixth Embodiment

[0127] Figure 7 The above is a flowchart illustrating the steps of a computer vision-based bridge health monitoring anomaly identification method according to the sixth embodiment of the present invention.

[0128] like Figure 7 As shown, the computer vision-based bridge health monitoring anomaly identification method includes the following steps:

[0129] The convolutional neural network, trained multiple times, is used as the health monitoring anomaly identification model for the bridge to be monitored.

[0130] Specifically, in this invention, different health monitoring anomaly identification models are designed for different bridges, and the health monitoring anomaly identification models are designed in various customized ways to achieve different structures for the health monitoring anomaly identification models of different bridges;

[0131] Three drones, positioned directly above, to the left, and to the right of the bridge to be monitored, respectively, captured video footage of the environment above, to the left, and to the right of the bridge within the current monitoring time segment. The distance from each of the three camera positions to the bridge was a set shooting distance.

[0132] Specifically, the three camera positions completed the all-round visual data collection of the bridge to be monitored, thus providing key data for the subsequent status identification of the bridge to be monitored;

[0133] Extract the sorting data of each frame of the video directly above the environment of the bridge to be monitored, the sorting data of each frame of the video directly to the left, and the sorting data of each frame of the video directly to the right within the current monitoring time segment, and use them as the three-camera visual sorting data corresponding to the current monitoring time segment.

[0134] Obviously, it is impractical to use every frame of the video from the top, left, and right sides of the environment of the bridge under monitoring within the current monitoring time segment as input for the subsequent model. This is because the amount of data in each frame is too large, and video processing is required to sort out the key visual data so that the less data-intensive three-camera visual sorting data can be used as input for the subsequent model.

[0135] Analyze the relevant capture information for each segment of the current monitoring time;

[0136] Specifically, the current monitoring time segment starts at the current moment. Therefore, the current monitoring time segment is a future time segment. The bridge structure anomaly types and the expected time of anomaly occurrence identified by subsequent synchronous intelligent identification are all future data.

[0137] Using the health monitoring anomaly identification model corresponding to the bridge to be monitored, based on the set shooting distance, the bridge type code value, the year of completion, the span length, the bridge design height and salt spray area marking, the duration of the current monitoring time segment, the three-camera visual sorting data corresponding to the current monitoring time segment, and the relevant capture information of each part of the current monitoring time segment, the model can intelligently identify various structural anomalies that will occur on the bridge to be monitored within the current monitoring time segment and the expected time of occurrence of each type of structural anomaly. The current monitoring time segment starts at the current time.

[0138] In this way, the synchronous intelligent identification of the future abnormal state types of the same bridge and the time of their occurrence is completed;

[0139] An anomaly warning mechanism, connected to a status recognition mechanism, is used to perform anomaly warning operations for the current monitoring time segment based on intelligent recognition results;

[0140] Specifically, abnormal warnings can be displayed on-site, transmitted wirelessly to a remote bridge monitoring server, or broadcast on-site, among other methods.

[0141] Among them, the analysis of each relevant captured information in the current monitoring time segment includes: the average cable tension, average ambient temperature, average ambient humidity, average on-site wind speed, on-site wind direction, total on-site traffic volume, and average load weight of the bridge under monitoring in the previous time segment before the current monitoring time segment as each relevant captured information in the current monitoring time segment.

[0142] For example, if the current time is 8:00 AM, the current monitoring time segment is from 8:00 AM to 8:10 AM, and the previous time segment was from 7:50 AM to 8:00 AM, then the relevant captured information for each segment of the current monitoring time segment is: the average cable tension, average ambient temperature, average ambient humidity, average on-site wind speed, on-site wind direction, total on-site traffic volume, and average load weight of the bridge under monitoring from 7:50 AM to 8:00 AM.

[0143] Among them, the abnormal early warning operation based on the intelligent recognition results for the current monitoring time segment includes: displaying on-site various structural abnormalities that will occur on the bridge to be monitored within the current monitoring time segment, as well as the expected occurrence time of various structural abnormalities;

[0144] For example, an on-site LED display array or an on-site LCD display array can be selected to display various structural anomalies that will occur on the bridge under monitoring within the current monitoring time segment, as well as the expected time of occurrence of various structural anomalies, as anomaly warning information on-site;

[0145] Each type of structural anomaly is one of the following: excessive deflection of the main beam, excessive displacement of the pier, excessive settlement of the pier, transverse cracks in the bridge deck, longitudinal cracks in the bridge deck, efflorescence of the bridge deck concrete, corrosion of the reinforcing steel, cracking of the pier concrete, and tilting of the guardrail.

[0146] Specifically, different types of structural anomalies have different numerical type codes.

[0147] The data sorted for each frame of each video consists of the depth value, grayscale value, vertical coordinate value and horizontal coordinate value of each pixel in the area occupied by the bridge to be monitored in that frame, as well as the grayscale gradient value, vertical coordinate value and horizontal coordinate value of each edge pixel in the area occupied by the bridge to be monitored in that frame.

[0148] For example, in each frame of each video, the pixel at the bottom left of the frame is taken as the origin of the horizontal coordinate system, the leftmost column of pixels in the frame is taken as the positive vertical axis of the horizontal coordinate system, and the bottom row of pixels in the frame is taken as the positive horizontal axis of the horizontal coordinate system, thus completing the construction of the horizontal coordinate system corresponding to the frame.

[0149] Among them, different bridge types correspond to different bridge type code values, and different bridge types include cable-stayed bridges, suspension bridges, arch bridges, railway bridges and urban overpass bridges. The salt spray area identifier of the bridge to be monitored is used to indicate whether the bridge to be monitored is in a salt spray area, and the resolution of the three drone aerial cameras is the same.

[0150] For example, different bridge types correspond to different bridge type codes, and different bridge types include cable-stayed bridges, suspension bridges, arch bridges, railway bridges and urban overpass bridges. The salt spray area identifier of the bridge to be monitored is used to indicate whether the bridge to be monitored is in a salt spray area, and the resolution of the three drone aerial cameras is the same, including: the resolution of the three drone aerial cameras can all be 8K resolution.

[0151] Among them, using a convolutional neural network trained multiple times as the health monitoring anomaly identification model for the bridge to be monitored includes: using a convolutional neural network trained multiple times as the health monitoring anomaly identification model for the bridge to be monitored, the convolutional neural network includes a single input layer, multiple convolutional layers, a single activation function layer, a single pooling layer and a single fully connected layer connected in sequence, the number of convolutional layers has the same numerical trend as the span length of the bridge to be monitored, and the number of training times is proportional to the completion year of the bridge to be monitored;

[0152] Specifically, the numerical variation trend of the number of convolutional layers and the span length of the bridge to be monitored is the same, which means that after the curve of the span length of the bridge to be monitored and the curve of the number of convolutional layers are normalized, the two processed curves can completely overlap.

[0153] For example, the span of the bridge to be monitored is 200 meters and the number of convolutional layers is 2; the span of the bridge to be monitored is 500 meters and the number of convolutional layers is 5; the span of the bridge to be monitored is 1000 meters and the number of convolutional layers is 10; the span of the bridge to be monitored is 1500 meters and the number of convolutional layers is 15, and so on.

[0154] For example, if the bridge to be monitored is 6 years old, the number of training sessions is 600; if the bridge to be monitored is 7 years old, the number of training sessions is 700; if the bridge to be monitored is 8 years old, the number of training sessions is 800; if the bridge to be monitored is 9 years old, the number of training sessions is 900, and so on.

[0155] In each training iteration of the convolutional neural network, various structural anomalies occurring on the bridge under monitoring within a specific past monitoring time segment, along with the times of occurrence of these anomalies, are used as the output of the convolutional neural network. The set shooting distance, bridge type code value, completion year, span length, bridge design height, salt spray area marker, occupation duration of the specified past monitoring time segment, the three-camera visual sorting data corresponding to the specified past monitoring time segment, and relevant capture information from the current monitoring time segment are used as the input of the convolutional neural network to complete this training.

[0156] Furthermore, in the computer vision-based bridge health monitoring anomaly identification system and method according to the present invention:

[0157] The numerical trend of the number of convolutional layers is the same as that of the span length of the bridge to be monitored, and the number of training sessions is proportional to the completion year of the bridge to be monitored. This includes: using a first numerical conversion formula to express the numerical conversion relationship between the number of convolutional layers and the span length of the bridge to be monitored, and using a second numerical conversion formula to express the numerical conversion relationship between the number of training sessions and the completion year of the bridge to be monitored.

[0158] The first numerical conversion formula is used to express the numerical conversion relationship between the number of convolutional layers and the span length of the bridge to be monitored, and the second numerical conversion formula is used to express the numerical conversion relationship between the number of training times and the completion year of the bridge to be monitored. In the first numerical conversion formula, the span length of the bridge to be monitored is the input value, and the number of convolutional layers is the output value.

[0159] The first numerical conversion formula is used to express the numerical conversion relationship between the number of convolutional layers and the span length of the bridge to be monitored, and the second numerical conversion formula is used to express the numerical conversion relationship between the number of training sessions and the completion year of the bridge to be monitored. The second numerical conversion formula also includes: in the second numerical conversion formula, the completion year of the bridge to be monitored is the input value and the number of training sessions is the output value.

[0160] For example, a programmable logic device can be selected to simultaneously simulate and model the first and second numerical conversion formulas. The programmable logic device selected can be a CPLD chip designed using VHDL language.

[0161] In the foregoing specification, the invention has been described with reference to specific embodiments. However, those skilled in the art will understand that various modifications and changes can be made without departing from the scope defined by the appended claims. Therefore, the specification and drawings should be considered illustrative rather than restrictive, and it is intended that all such modifications be included within the scope of the invention.

Claims

1. A bridge health monitoring anomaly identification system based on computer vision, characterized in that, The system includes: An object construction mechanism is used to use a convolutional neural network that has been trained multiple times as a health monitoring anomaly identification model for the bridge to be monitored. The directional acquisition mechanism is used to collect video from the top, left, and right sides of the environment of the bridge under monitoring within the current monitoring time segment by using three drone aerial cameras located at three shooting positions directly above, to the left, and to the right of the bridge under monitoring. The distance from the three shooting positions to the bridge under monitoring is a set shooting distance. The video processing mechanism, connected to the directional acquisition mechanism, is used to extract the sorting data of each frame of the video directly above the environment of the bridge to be monitored, the sorting data of each frame of the video directly to the left, and the sorting data of each frame of the video directly to the right within the current monitoring time segment, so as to serve as the three-camera visual sorting data corresponding to the current monitoring time segment. Information analysis agency, used to analyze each relevant captured information segment of the current monitoring time period; The status recognition mechanism is connected to the object construction mechanism, video processing mechanism, and information analysis mechanism, respectively. It is used to identify various structural anomalies that will occur on the bridge under monitoring and the expected time of occurrence of various structural anomalies in the current monitoring time segment, based on the set shooting distance, the bridge type code value, the year of completion, the span length, the bridge design height and salt spray area marking, the duration of the current monitoring time segment, the three-camera visual sorting data corresponding to the current monitoring time segment, and the relevant captured information of the current monitoring time segment. The current monitoring time segment starts from the current time. An anomaly warning mechanism, connected to a status recognition mechanism, is used to perform anomaly warning operations for the current monitoring time segment based on intelligent recognition results.

2. The bridge health monitoring anomaly identification system based on computer vision as described in claim 1, characterized in that: The analysis of relevant captured information for the current monitoring time segment includes: the average cable tension, average ambient temperature, average ambient humidity, average on-site wind speed, on-site wind direction, total on-site traffic volume, and average load weight of the bridge under monitoring in the previous time segment before the current monitoring time segment as relevant captured information for the current monitoring time segment. Among them, the abnormal early warning operation based on the intelligent recognition results for the current monitoring time segment includes: displaying on-site various structural abnormalities that will occur on the bridge to be monitored within the current monitoring time segment, as well as the expected occurrence time of various structural abnormalities; Each type of structural anomaly is one of the following: excessive deflection of the main beam, excessive displacement of the pier, excessive settlement of the pier, transverse cracks in the bridge deck, longitudinal cracks in the bridge deck, efflorescence of the bridge deck concrete, corrosion of the reinforcing steel, cracking of the pier concrete, and tilting of the guardrail. The data sorted for each frame of each video consists of the depth value, grayscale value, vertical coordinate value and horizontal coordinate value of each pixel in the area occupied by the bridge to be monitored in that frame, as well as the grayscale gradient value, vertical coordinate value and horizontal coordinate value of each edge pixel in the area occupied by the bridge to be monitored in that frame. Different bridge types correspond to different bridge type codes, and the different bridge types include cable-stayed bridges, suspension bridges, arch bridges, railway bridges and urban overpass bridges. The salt fog area identifier of the bridge to be monitored is used to indicate whether the bridge to be monitored is in a salt fog area, and the resolution of the three drone aerial cameras is the same.

3. The bridge health monitoring anomaly identification system based on computer vision as described in claim 2, characterized in that: Using a convolutional neural network trained multiple times as a health monitoring anomaly identification model for the bridge to be monitored includes: using a convolutional neural network trained multiple times as a health monitoring anomaly identification model for the bridge to be monitored, the convolutional neural network includes a single input layer, multiple convolutional layers, a single activation function layer, a single pooling layer and a single fully connected layer connected in sequence, the number of convolutional layers has the same numerical trend as the span length of the bridge to be monitored, and the number of training times is proportional to the completion year of the bridge to be monitored; In each training iteration of the convolutional neural network, various structural anomalies occurring on the bridge under monitoring within a specific past monitoring time segment, along with the times of occurrence of these anomalies, are used as the output of the convolutional neural network. The set shooting distance, bridge type code, completion year, span length, bridge design height, salt spray area markers, the duration of the specified past monitoring time segment, the three-camera visual sorting data corresponding to the specified past monitoring time segment, and relevant capture information from the current monitoring time segment are used as the inputs to the convolutional neural network to complete this training.

4. The bridge health monitoring anomaly identification system based on computer vision as described in claim 3, characterized in that, The system also includes: The wireless transmission mechanism, connected to the status identification mechanism, is used to synchronously and wirelessly transmit the various structural anomalies that will occur on the bridge under monitoring within the current monitoring time segment, as well as the expected occurrence time of each structural anomaly, to the remote bridge monitoring server. The method of synchronously and wirelessly transmitting various structural anomalies that will occur on the bridge under monitoring during the current monitoring time segment, as well as the expected occurrence time of each structural anomaly, to a remote bridge monitoring server includes: packaging all kinds of structural anomalies that will occur on the bridge under monitoring during the current monitoring time segment, as well as the expected occurrence time of each structural anomaly, into the same network data packet, and then wirelessly transmitting the network data packet to the remote bridge monitoring server.

5. The bridge health monitoring anomaly identification system based on computer vision as described in claim 3, characterized in that, The system also includes: The acoustic early warning mechanism, connected to the status recognition mechanism, is used to receive various structural anomalies that will occur on the bridge under monitoring within the current monitoring time segment, as well as the estimated time of occurrence of each structural anomaly. The acoustic early warning mechanism is also used to broadcast on-site various structural anomalies that will occur on the bridge under monitoring within the current monitoring time segment, as well as the expected time of occurrence of each structural anomaly.

6. The bridge health monitoring anomaly identification system based on computer vision as described in claim 3, characterized in that, The system also includes: The synchronous control mechanism, connected to the directional acquisition mechanism, is used to perform synchronous control of three drone aerial photography devices located at three camera positions directly above, to the left and to the right of the bridge to be monitored. The synchronous control of three drones positioned at three camera locations—directly above, to the left of, and to the right of the bridge to be monitored—includes controlling the three drones to simultaneously perform shooting actions to collect video footage from directly above, to the left of, and to the right of the environment surrounding the bridge to be monitored within the current monitoring time segment.

7. The bridge health monitoring anomaly identification system based on computer vision as described in claim 3, characterized in that, The system also includes: The timing service provider is connected to the status identification provider and the directional data acquisition provider respectively, and is used to provide the timing services required by the status identification provider and the directional data acquisition provider respectively. The timing service mechanism is connected to the status identification mechanism and the directional acquisition mechanism respectively, and is used to provide the status identification mechanism and the directional acquisition mechanism with their respective timing services. The timing service mechanism provides the status identification mechanism and the directional acquisition mechanism with their respective timing services by providing a reference clock signal with a square waveform of a set frequency.

8. The bridge health monitoring anomaly identification system based on computer vision as described in any one of claims 3-7, characterized in that: Based on the gray value distribution range corresponding to the bridge, the area of ​​the image occupied by the bridge to be monitored in each frame is identified. The gray value distribution range corresponding to the bridge is formed by limiting the upper limit gray value and the lower limit gray value corresponding to the bridge, and the upper limit gray value corresponding to the bridge is greater than the lower limit gray value corresponding to the bridge. Specifically, the method involves identifying the area occupied by the bridge to be monitored in each frame based on the grayscale value distribution range corresponding to the bridge. The grayscale value distribution range corresponding to the bridge is a value distribution range formed by limiting the upper limit grayscale value and the lower limit grayscale value corresponding to the bridge, and the upper limit grayscale value corresponding to the bridge is greater than the lower limit grayscale value corresponding to the bridge. This includes: taking the pixels in each frame whose grayscale value is within the grayscale value distribution range corresponding to the bridge as the suspected bridge to be monitored pixels in that frame; removing isolated pixels from each of the suspected bridge to be monitored pixels in that frame to obtain the remaining suspected bridge to be monitored pixels; and fitting the remaining suspected bridge to be monitored pixels to obtain the area occupied by the bridge to be monitored in that frame.

9. The bridge health monitoring anomaly identification system based on computer vision as described in any one of claims 3-7, characterized in that: The average cable tension, average ambient temperature, average ambient humidity, average on-site wind speed, on-site wind direction, total on-site traffic volume, and average load weight of the bridge under monitoring in the previous time segment are used as the relevant captured information of the bridge under monitoring in the current monitoring time segment. This includes: the arithmetic mean of the remaining cable tensions obtained after removing the maximum and minimum values ​​of each cable tension corresponding to each time segment at even intervals in the current monitoring time segment; the average cable tension of the bridge under monitoring in the previous time segment; and the arithmetic mean of the remaining on-site wind speeds obtained after removing the maximum and minimum values ​​of each on-site wind speed corresponding to each time segment at even intervals in the current monitoring time segment. The data captured for the bridge under monitoring in the current monitoring time segment includes the average cable tension, average ambient temperature, average ambient humidity, average on-site wind speed, on-site wind direction, total on-site traffic volume, and average load weight within the time segment preceding the current monitoring time segment. It also includes the arithmetic mean of the remaining ambient temperatures obtained after removing the maximum and minimum values ​​from the ambient temperatures at evenly spaced intervals within the current monitoring time segment, and the arithmetic mean of the remaining ambient humidity values ​​obtained after removing the maximum and minimum values ​​from the ambient humidity values ​​at evenly spaced intervals within the current monitoring time segment, which is used as the average ambient temperature of the bridge under monitoring in the current monitoring time segment. The data captured for each relevant segment of the bridge under monitoring in the current monitoring time segment includes the average cable tension, average ambient temperature, average ambient humidity, average on-site wind speed, on-site wind direction indicator, total on-site traffic volume, and average load weight of the bridge under monitoring in the previous time segment. It also includes the arithmetic mean of the remaining multiple bridge load weights obtained by removing the maximum and minimum values ​​from each bridge load weight at evenly spaced intervals within the current monitoring time segment. This arithmetic mean is used as the average total load of the bridge under monitoring in the current monitoring time segment, with different wind direction indicators for different wind directions.

10. A method for anomaly identification in bridge health monitoring based on computer vision, characterized in that, The method includes: The convolutional neural network, trained multiple times, is used as the health monitoring anomaly identification model for the bridge to be monitored. Three drones, positioned directly above, to the left, and to the right of the bridge to be monitored, respectively, captured video footage of the environment above, to the left, and to the right of the bridge within the current monitoring time segment. The distance from each of the three camera positions to the bridge was a set shooting distance. Extract the sorting data of each frame of the video directly above the environment of the bridge to be monitored, the sorting data of each frame of the video directly to the left, and the sorting data of each frame of the video directly to the right within the current monitoring time segment, and use them as the three-camera visual sorting data corresponding to the current monitoring time segment. Analyze the relevant capture information for each segment of the current monitoring time; Using the health monitoring anomaly identification model corresponding to the bridge to be monitored, based on the set shooting distance, the bridge type code value, the year of completion, the span length, the bridge design height and salt spray area marking, the duration of the current monitoring time segment, the three-camera visual sorting data corresponding to the current monitoring time segment, and the relevant capture information of each part of the current monitoring time segment, the model can intelligently identify various structural anomalies that will occur on the bridge to be monitored within the current monitoring time segment and the expected time of occurrence of each type of structural anomaly. The current monitoring time segment starts at the current time. Based on the intelligent recognition results, perform abnormal early warning operations for the current monitoring time segment.

Images