A bridge health early warning method and system based on computer vision

Abstract

This application provides a bridge health early warning method and system based on computer vision. It can acquire state monitoring images of a target bridge to determine the actual displacement of each frame; construct a bridge model to determine the static deflection of the bridge model under preset load conditions; determine abnormal static impacts of the target bridge under preset loads by analyzing all actual displacements and the static deflection; extract vehicle motion information to construct the bridge displacement trajectories when different vehicles pass over the bridge; perform load analysis on various displacement trajectories using the static deflection to obtain the motion impact response of the load under various displacement trajectories; determine the state of load displacement under different load conditions based on abnormal static impacts and all motion impact responses; and output health early warning signals corresponding to different load displacement states. Using the scheme of this application, continuous monitoring of bridges can be achieved, reducing the simulation difficulty of finite element analysis.

Description

Technical Field

[0001] This application relates to the field of status early warning technology, and in particular to a bridge health early warning method and system based on computer vision. Background Technology

[0002] Condition monitoring is a key technology applied across various industries, particularly in industrial and infrastructure monitoring. It involves using methods such as sensors, data analysis, and machine learning to monitor and assess the current state of a system or structure, and to predict its future performance and potential risks. The development and application of condition monitoring technology is of great significance for improving system safety, reliability, and efficiency, and is one of the key technologies for achieving intelligent management and maintenance. With the advancement of IoT, big data, and artificial intelligence technologies, condition monitoring technology will continue to develop, providing more accurate and efficient solutions for various industries.

[0003] Computer vision-based bridge health early warning utilizes image processing and analysis to monitor the visual information of bridges in order to assess their structural condition and safety. Image processing is the core of this technology, involving steps such as image enhancement, segmentation, and feature extraction. The processed image data is then analyzed to monitor the bridge's health status in real time. Relevant thresholds are set to detect potential damage or abnormal changes in the bridge, triggering alarms and providing corresponding assessment reports. However, in existing technologies, due to insufficient application of long-term monitoring techniques, most highway bridges struggle to accumulate continuous displacement data during their operational period. This increases the difficulty of simulating actual bridge conditions using finite element analysis, making it impossible to provide corresponding early warning values ​​for bridge structural deformation monitoring. Therefore, how to achieve continuous bridge monitoring to reduce the simulation difficulty of finite element analysis has become a challenge for the industry. Summary of the Invention

[0004] This application provides a bridge health early warning method and system based on computer vision, which can realize continuous monitoring of bridges to reduce the simulation difficulty of finite element analysis.

[0005] In a first aspect, this application provides a bridge health early warning method based on computer vision, comprising the following steps:

[0006] Automatically acquire status monitoring images of the target bridge, and then determine the actual displacement of the target bridge in each frame of the status monitoring image;

[0007] A bridge model is constructed based on the design information of the target bridge, and then the static deflection of the bridge model under preset load conditions is determined.

[0008] The abnormal static impact of the target bridge under the preset load is determined by all actual displacements and the static deflection.

[0009] Extract the motion information of vehicles on the target bridge, and then construct the displacement trajectory of the bridge when different vehicle motion states pass through the bridge. Perform load analysis on various displacement trajectories through the static deflection to obtain the motion impact response of the load under various displacement trajectories.

[0010] Based on the abnormal static impact and all motion impact responses, the load displacement state of the target bridge under different load conditions is determined, and then a health warning signal corresponding to the different load displacement states is output.

[0011] In some embodiments, determining the actual displacement of the target bridge in each frame of the status monitoring image specifically includes:

[0012] Obtain a template image of the target bridge within a specified area;

[0013] Select one frame of the status monitoring image as the selected status monitoring image;

[0014] The selected status monitoring image is matched with the template image to obtain a matching image of the selected status monitoring image;

[0015] Multiple feature point pairs are determined on the matched image;

[0016] The pixel motion of the selected state monitoring image relative to the template image is determined based on all feature point pairs;

[0017] Based on the size information of the target bridge, the pixel motion is converted into the actual displacement of the target bridge;

[0018] Continue to convert the pixel motion of the remaining state monitoring images in the marked area into the actual displacement of the target bridge.

[0019] In some embodiments, determining the abnormal static impact of the target bridge under a preset load by using all actual displacements and the static deflection specifically includes:

[0020] The displacement deviation of each frame of the state monitoring image is determined by all actual displacements and the static deflection.

[0021] The abnormal static impact of the target bridge under the preset load is determined based on all displacement deviations.

[0022] In some embodiments, extracting the motion information of vehicles on the target bridge and then constructing the displacement trajectory of the bridge when different vehicle motion states pass through the bridge specifically includes:

[0023] Acquire vehicle motion information in each frame of the status monitoring image;

[0024] A dynamic load model of the target bridge is constructed based on the motion information;

[0025] The dynamic load model is used to determine the displacement trajectory of the bridge when different vehicle motion states pass over the bridge.

[0026] In some embodiments, determining the bridge displacement trajectory when different vehicle motion states pass over the bridge using the dynamic load model specifically includes:

[0027] Determine all load differences of the target bridge based on the motion information;

[0028] Determine the motion states of various vehicles on the target bridge by measuring all load differences;

[0029] The displacement trajectory of the bridge when different vehicle motion states pass through the bridge is determined by the dynamic load model and all vehicle motion states.

[0030] In some embodiments, load analysis of various displacement trajectories using the static deflection to obtain the kinematic impact response of the load under various displacement trajectories specifically includes:

[0031] Select a displacement trajectory as the selected displacement trajectory;

[0032] Analyze the reference displacement of the target bridge under the selected displacement trajectory;

[0033] The motion impact response of the load under the selected displacement trajectory is determined by the reference displacement and the static deflection.

[0034] Continue to determine the motion impact response of the load under the remaining displacement trajectory.

[0035] In some embodiments, determining the load displacement state of the target bridge under different load conditions based on the abnormal static impact and all kinematic impact responses specifically includes:

[0036] Obtain the static deflection of the target bridge;

[0037] The maximum displacement of the target bridge under static load is determined based on the abnormal static impact and the static deflection.

[0038] The maximum displacement of the target bridge under different load conditions is determined based on all motion impact responses and the static deflection.

[0039] The state of load displacement under different load conditions is determined based on the maximum displacement corresponding to each motion impact response and the maximum displacement of the target bridge under static load.

[0040] Secondly, this application provides a bridge health early warning system based on computer vision, comprising:

[0041] The acquisition module is used to automatically acquire status monitoring images of the target bridge, and then determine the actual displacement of the target bridge in each frame of the status monitoring image.

[0042] The processing module is used to construct a bridge model based on the design information of the target bridge, and then determine the static deflection of the bridge model under preset load conditions.

[0043] The processing module is also used to determine the abnormal static impact of the target bridge under the preset load by using all actual displacements and the static deflection;

[0044] The processing module is also used to extract the motion information of vehicles on the target bridge, and then construct the displacement trajectory of the bridge when different vehicle motion states pass through the bridge. The static deflection is used to perform load analysis on various displacement trajectories to obtain the motion impact response of the load under various displacement trajectories.

[0045] The execution module is used to determine the load displacement state of the target bridge under different load conditions based on the abnormal static impact and all motion impact responses, and then outputs health warning signals corresponding to different load displacement states.

[0046] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described computer vision-based bridge health early warning method.

[0047] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described computer vision-based bridge health early warning method.

[0048] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0049] The bridge health early warning method and system based on computer vision provided in this application automatically acquires state monitoring images of the target bridge, thereby determining the actual displacement of the target bridge in each frame of the state monitoring image; constructs a bridge model based on the design information of the target bridge, thereby determining the static deflection of the bridge model under preset load conditions; determines the abnormal static impact of the target bridge under preset load by using all actual displacements and the static deflection; extracts the motion information of vehicles on the target bridge, thereby constructing the displacement trajectory of the bridge when different vehicle motion states pass through the bridge; performs load analysis on various displacement trajectories using the static deflection to obtain the motion impact response of the load under various displacement trajectories; determines the load displacement state of the target bridge under different load conditions based on the abnormal static impact and all motion impact responses, and then outputs health early warning signals corresponding to different load displacement states.

[0050] Therefore, this application firstly achieves long-term, continuous, and real-time displacement data acquisition of bridges through image monitoring technology, thus avoiding the reliance on discrete time-point data acquisition in traditional monitoring technologies. This provides a richer continuous data source, offering more accurate input conditions for finite element analysis. Furthermore, based on feature matching of each frame of the state monitoring image, continuous displacement information (actual displacement) is obtained in each frame. Finite element modeling is then performed using this continuous displacement information to obtain the bridge model, thereby reducing the difficulty of simulating the actual working conditions of bridges through finite element analysis. Secondly, the bridge model determines the bridge state under static loads (abnormal static impact) and under different load conditions (kinematic impact response). Based on the abnormal static impact and all kinematic impact responses, the load displacement state of the target bridge under different load conditions is determined. This quantifies the deformation and displacement of the bridge under dynamic loads, allowing for real-time monitoring of the various load displacement states. In the event of health anomalies, an early warning signal is sent to the target bridge's health monitoring center. In summary, this application enables continuous monitoring of bridges, reducing the simulation difficulty of finite element analysis. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating a computer vision-based bridge health early warning method according to some embodiments of this application;

[0052] Figure 2 This is a flowchart illustrating the determination of a displacement trajectory according to some embodiments of this application;

[0053] Figure 3 This is a flowchart illustrating the process of determining the state of load displacement according to some embodiments of this application;

[0054] Figure 4 This is a structural schematic diagram of a computer vision-based bridge health early warning system according to some embodiments of this application;

[0055] Figure 5 This is an internal structural diagram of a computer device that implements a computer vision-based bridge health early warning method according to some embodiments of this application. Detailed Implementation

[0056] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] refer to Figure 1The figure is a flowchart illustrating a computer vision-based bridge health early warning method according to some embodiments of this application. The computer vision-based bridge health early warning method 100 mainly includes the following steps:

[0058] In step 101, the status monitoring images of the target bridge are automatically acquired, and then the actual displacement of the target bridge in each frame of the status monitoring image is determined.

[0059] Preferably, the target bridge's condition monitoring images can be automatically acquired using a camera device. It should be noted that the camera device is a high-definition industrial camera, which has high resolution and can capture subtle movements and deformations of the bridge. It is used for high-precision displacement and deformation monitoring of designated areas of the target bridge (such as mid-span, supports, and cracked areas). In this application, after capturing video of the marked area using the high-definition industrial camera, each frame of the video is used as a condition monitoring image of the target bridge. Furthermore, the marked area represents a key part of the target bridge prone to displacement (such as mid-span, supports, and cracked areas). In a preferred embodiment, measuring point markers can be set at the marked area, serving as target markers in computer vision technology. This provides known reference points during image processing and analysis, helping the system to more accurately track and measure the displacement changes of the target bridge, thereby improving the accuracy and reliability of the monitoring data.

[0060] In some embodiments, determining the actual displacement of the target bridge in each frame of the status monitoring image can be achieved by the following steps:

[0061] Obtain a template image of the target bridge within a specified area;

[0062] Select one frame of the status monitoring image as the selected status monitoring image;

[0063] The selected status monitoring image is matched with the template image to obtain a matching image of the selected status monitoring image;

[0064] Multiple feature point pairs are determined on the matched image;

[0065] The pixel motion of the selected state monitoring image relative to the template image is determined based on all feature point pairs;

[0066] Based on the size information of the target bridge, the pixel motion is converted into the actual displacement of the target bridge;

[0067] Continue to convert the pixel motion of the remaining state monitoring images in the marked area into the actual displacement of the target bridge.

[0068] In specific implementation, image processing software (e.g., OpenCV) can be used to process the status monitoring image of the target bridge, extract the image of the specified area and optimize it into a template image. In other embodiments, other methods can also be used to determine the template image, which is not limited here. It should be noted that the template image is a standard image taken under specific conditions and is used as the basis for comparison to realize feature matching and displacement calculation of the status monitoring image.

[0069] In specific implementation, matching the selected status monitoring image with the template image to obtain the matching image of the selected status monitoring image can be achieved in the following way: First, the position where the coordinates of the upper left corner of the template image coincide with the coordinates of the upper left corner of the selected status monitoring image is taken as the initial position of the template image. Then, the width and height of the template image are obtained. Using the width and height as input, the normalized correlation coefficient matching calculation function in the prior art is used to determine the correlation value between the template image and the selected status monitoring image at the initial position. Next, the template image is continuously slid so that the upper left corner vertex of the template image traverses every pixel in the selected status monitoring image, thereby obtaining multiple correlation values. Finally, when the result of the correlation value is 1, the area that the template image has moved to is taken as the matching position of the template image in the selected status monitoring image. When the result of the correlation value is not 1, no processing is performed. Thus, the image after matching is taken as the matching image of the selected status monitoring image. In other embodiments, other methods can also be used to achieve this, which are not limited here.

[0070] It should be noted that the matching image mentioned in this application refers to the region found in the selected status monitoring image that corresponds to the template image. This region contains parts with similar features to the template image. In addition, the correlation value characterizes the degree of similarity between different regions in the selected status monitoring image and the template image. The larger the correlation value, the higher the similarity between the selected region and the template image. The smaller the correlation value, the lower the similarity between the selected region and the template image. The correlation value ranges from (0, 1).

[0071] In specific implementation, determining multiple feature point pairs on the matching image can be achieved in the following way: First, the second derivative of the matching image is calculated to construct the Hessian matrix of the matching image. Then, a Gaussian filter is used to smooth the image at different scales to generate images at multiple scales. The Hessian matrix is ​​calculated for each scale image to construct a scale space. Subsequently, the feature points in the matching image are located by the positions of the extreme points of the Hessian matrix in the scale space. The principal direction of each feature point in the neighborhood region is calculated using the Haar wavelet response, which is a technique in the prior art. Next, feature descriptors for each feature point are constructed based on the pixel intensity differences in the neighborhood of each feature point. Finally, the feature descriptors of each feature point are used as input, and the FLANN algorithm, which is a technique in the prior art, is used to match all feature points to obtain multiple feature point pairs. In other embodiments, other methods can also be used to determine the feature points, which are not limited here.

[0072] It should be noted that the feature points mentioned in this application refer to the pixel points with significant features extracted from the matching image. In addition, the two feature points in the feature point pair are a pixel point pair consisting of a feature point identified in a template image and a feature point in the matching image that corresponds to the template image.

[0073] In specific implementation, the pixel motion of the selected state monitoring image relative to the template image can be determined based on all feature point pairs in the following way: the sum of the distances between corresponding two feature points in all feature point pairs divided by the total number of feature point pairs in the matching image is taken as the pixel motion of the selected state monitoring image relative to the template image. It should be noted that the distance can be determined by taking the upper left corner of the selected state monitoring image as the origin, taking the horizontal direction to the right along the origin as the X-axis, and the vertical downward direction as the Y-axis, and assigning coordinates to each pixel in the selected image, thereby using the Euclidean distance algorithm in the prior art to determine the pixel distance between corresponding two feature points in all feature point pairs. In other embodiments, other methods can also be used to determine the distance, which is not limited here.

[0074] It should be noted that the pixel motion refers to the movement distance between feature points in the selected state monitoring image corresponding to the template image. It reflects the image changes caused by bridge deformation or other factors. The greater the pixel motion, the greater the image changes caused by bridge deformation or other factors. The smaller the pixel motion, the smaller the image changes caused by bridge deformation or other factors.

[0075] In specific implementation, the conversion of pixel motion into actual displacement of the target bridge based on its size information can be achieved in the following way: First, the actual length of the target bridge is obtained based on its size information. Then, the pixel distance between pixels in the selected state monitoring image is obtained. Subsequently, the value of dividing the actual length by the pixel distance is used as a conversion coefficient. Finally, the conversion coefficient is multiplied by the result of the pixel motion corresponding to the selected state monitoring image to obtain the actual displacement of the target bridge. In other embodiments, other methods can also be used to determine this, which are not limited here. It should be noted that the conversion coefficient is the ratio between the pixel distance in the image and the actual length of the target bridge, used to convert the pixel motion measured in the image into actual physical displacement. In addition, the actual displacement refers to the physical displacement of the target bridge observed during the monitoring process.

[0076] In step 102, a bridge model is constructed based on the design information of the target bridge, and then the static deflection of the bridge model under preset load conditions is determined.

[0077] In specific implementation, after obtaining the design information of the target bridge, the transverse arch wave of the target bridge can be simplified into a straight beam. The columns on the arch are rigidly connected to the main arch and the web arch, while the filler on the arch is elastically connected to the main arch. CAD software can be used to create the geometric model of the bridge. The geometric model can be imported into finite element analysis software (such as ANSYS, ABAQUS, SAP2000) and meshed to establish the finite element model of the target bridge. This finite element model is used as the bridge model in this application. In addition, after determining the bridge model of the target bridge, a preset load can be set on the bridge model to determine the displacement in the marked area corresponding to the bridge model. The displacement is used as the static deflection of the bridge model in the marked area under the preset load. Other methods can also be used in other embodiments, which are not limited here.

[0078] In some embodiments, determining the static deflection of the bridge model under preset load conditions can be achieved by the following steps:

[0079] Obtain the preset load of the target bridge;

[0080] The displacement field under the preset load condition is determined by the preset load and the bridge model.

[0081] The static deflection of the bridge model under preset load conditions is generated based on the displacement field.

[0082] In specific implementation, the constant load of the target bridge (including the bridge's own weight, the weight of permanent facilities, and other loads that continuously act on the bridge) can be used as the preset load in this application. In other embodiments, other methods can also be used to obtain the load, which are not limited here.

[0083] It should be noted that, in this application, after selecting the Static Structural Analysis (SSA) module in the finite element analysis software, the displacement field of the bridge model under the preset load conditions is generated. The displacement result of the marked area in the model is determined by selecting the marked area (usually the mid-span, key node, or a certain structural part of the bridge) through the software interface. The vertical displacement of the bridge model in the marked area is taken as the static deflection of the bridge model under the preset load conditions. In other embodiments, other methods can be used to determine the displacement, which is not limited here.

[0084] In step 103, the abnormal static impact of the target bridge under the preset load is determined by all actual displacements and the static deflection.

[0085] In some embodiments, determining the abnormal static impact of the target bridge under a preset load by means of all actual displacements and the static deflection can be achieved by the following steps:

[0086] The displacement deviation of each frame of the state monitoring image is determined by all actual displacements and the static deflection.

[0087] The abnormal static impact of the target bridge under the preset load is determined based on all displacement deviations.

[0088] It should be noted that the displacement deviation represents the difference between the actual displacement (actual displacement) and the expected displacement (static deflection) of the target bridge under static load within a specific time frame. It characterizes the degree of difference between the actual displacement of the target bridge and its static deflection under static load. The larger the displacement deviation, the higher the degree of difference between the actual displacement of the target bridge and its static deflection under static load. The smaller the displacement deviation, the lower the degree of difference between the actual displacement of the target bridge and its static deflection under static load. As a preferred embodiment, the displacement deviation of each frame of the status monitoring image can be determined by the following method: the difference between the actual displacement and the static deflection corresponding to each frame of the status monitoring image is used as the displacement deviation of each frame of the status monitoring image. In other embodiments, other methods can also be used to determine it, which are not limited here.

[0089] In practice, the abnormal static impact of the target bridge under the preset load can be determined by the following method based on all displacement deviations: First, determine the average value of all displacement deviations. Then, divide the result by the static deflection to obtain the abnormal static impact of the target bridge under the preset load.

[0090] It should be noted that, in the context of bridge health monitoring in this application, static impact refers to the static response of a bridge under preset load conditions. This response may include displacement, deformation, etc., caused by the load. The abnormal static impact described in this application is a mapping value of the difference between the actual structural response and the design expectation of the target bridge under preset load conditions. The larger the abnormal static impact, the higher the difference between the actual structural response and the design expectation of the target bridge under preset load conditions; the smaller the abnormal static impact, the lower the difference between the actual structural response and the design expectation of the target bridge under preset load conditions. This application determines the average value of all displacement deviations and then divides the result by the static deflection to obtain the abnormal static impact of the target bridge under preset load, that is, it quantifies the difference between the actual structural response and the design expectation, thereby providing data support for assessing the health status of the bridge.

[0091] In step 104, the motion information of vehicles on the target bridge is extracted, and then the displacement trajectory of the bridge when different vehicle motion states pass through the bridge is constructed. The static deflection is used to perform load analysis on various displacement trajectories to obtain the motion impact response of the load under various displacement trajectories.

[0092] In some embodiments, the extraction of vehicle motion information on the target bridge, and the subsequent construction of the bridge displacement trajectory when different vehicle motion states pass over the bridge, are achieved in the following manner:

[0093] Acquire vehicle motion information in each frame of the status monitoring image;

[0094] A dynamic load model of the target bridge is constructed based on the motion information;

[0095] The dynamic load model is used to determine the displacement trajectory of the bridge when different vehicle motion states pass over the bridge.

[0096] Preferably, this application uses computer vision technology (e.g., YOLO, OpenCV, etc.) to process the status monitoring image to identify and extract the vehicles in the status monitoring image, and records the position information (coordinates in the status monitoring image) and timestamp of each vehicle as the motion information of the vehicle on the status monitoring image, thereby obtaining the motion information of the vehicle on each frame of the status monitoring image. In other embodiments, other methods can also be used to determine the motion information, which is not limited here.

[0097] In specific implementation, the dynamic load model of the target bridge can be constructed based on the motion information in the following way: First, after obtaining the mass of different vehicles in all state monitoring images, the vehicle information and the motion information are input into the bridge model, and the obtained model is used as the dynamic load model of the target bridge. In other embodiments, other methods can also be used to determine the load model, which is not limited here.

[0098] In some embodiments, reference Figure 2 As shown in the figure, this is a flowchart illustrating the determination of displacement trajectory in some embodiments of this application. The determination of the bridge's displacement trajectory when different vehicle motion states pass over the bridge using the dynamic load model can be achieved through the following steps:

[0099] First, in step 1041, all load differentials of the target bridge are determined based on the motion information.

[0100] Then, in 1042, the motion states of various vehicles on the target bridge are determined by all load differentials;

[0101] Finally, in section 1043, the displacement trajectory of the bridge when different vehicle motion states pass through the bridge is determined by the dynamic load model and all vehicle motion states.

[0102] In specific implementation, determining the load differences of the target bridge based on the motion information can be achieved in the following way: First, obtain the vehicle position information in all adjacent frame status monitoring images based on the motion information. Then, select a pair of adjacent frame status monitoring images. For each vehicle in the pair of adjacent frame status monitoring images, extract its position coordinates from the two frame status monitoring images. Then, use the Euclidean distance algorithm in the prior art to calculate the vehicle's displacement between the two frame status monitoring images. Subsequently, calculate the difference in timestamps between the pair of adjacent frame status monitoring images, and multiply the reciprocal of the result by the vehicle's displacement between the two frame status monitoring images to obtain the vehicle's speed. The speed of all vehicles in the frame state monitoring image is then taken as the average speed of all vehicles in the adjacent frame state monitoring images. Next, after determining the vehicle speeds between all adjacent frame state monitoring images, the absolute difference between every two vehicle speeds is determined, and all absolute differences are used as the load difference degree among all vehicles. It should be noted that the load difference degree characterizes the degree of deviation of the corresponding vehicle's load on the target bridge in adjacent state monitoring images. The larger the load difference degree, the higher the degree of deviation of the corresponding vehicle's load on the target bridge in adjacent state monitoring images; the smaller the load difference degree, the lower the degree of deviation of the corresponding vehicle's load on the target bridge in adjacent state monitoring images.

[0103] It should be noted that this application is based on the different dynamic load effects of vehicles on bridges at different speeds. The load situation of the target bridge is divided into multiple load modes according to different vehicle speeds. That is, when vehicles pass through the bridge at different speeds, different impact coefficients and dynamic loads are generated. The faster the vehicle speed, the stronger the instantaneous impact and dynamic effect on the bridge, which will lead to an increase in the bridge displacement, acceleration, vibration frequency, and other responses. Conversely, the lower the speed, the smaller the bridge displacement, acceleration, vibration frequency, and other responses. Therefore, as a preferred embodiment, the determination of multiple vehicle motion states on the target bridge through all load differences can be achieved in the following way: after using all load differences as similarity indicators in the K-means clustering algorithm in the prior art, the K-means clustering algorithm in the prior art is used to divide all vehicle speeds into multiple vehicle speed clusters, and then each vehicle speed cluster is used as a vehicle motion state on the target bridge, thereby obtaining multiple vehicle motion states on the target bridge. In other embodiments, other methods can also be used to determine them, which are not limited here.

[0104] It should be noted that the displacement trajectory mentioned in this application refers to the displacement, deformation, or movement path of the target bridge structure under the corresponding load mode over time. The displacement trajectory usually reflects the structural change trend of the bridge when subjected to dynamic loads in a specified area (such as mid-span, support point, node, etc.). The smoother the displacement trajectory, the slower the change trend of the target bridge structure; the steeper the displacement trajectory, the faster the change trend of the target bridge structure. In specific implementation, the displacement trajectory of the bridge when passing through the bridge under different vehicle motion states can be determined by the following method: First, select a vehicle motion state and input the speeds of all vehicles in this vehicle motion state into the dynamic load model. Then, apply constraints to the dynamic load model and solve the dynamic equations to obtain the displacement of the target bridge in a specified area (key positions of the bridge structure such as mid-span or support point) under different vehicle speeds. The set of all displacements is taken as the displacement trajectory of the target bridge under this vehicle motion state. Repeat the above steps to determine the displacement trajectory of the target bridge under the remaining vehicle motion states.

[0105] In some embodiments, the following steps can be used to perform load analysis on various displacement trajectories using the static deflection to obtain the kinematic impact response of the load under various displacement trajectories:

[0106] Select a displacement trajectory as the selected displacement trajectory;

[0107] Analyze the reference displacement of the target bridge under the selected displacement trajectory;

[0108] The motion impact response of the load under the selected displacement trajectory is determined by the reference displacement and the static deflection.

[0109] Continue to determine the motion impact response of the load under the remaining displacement trajectory.

[0110] It should be noted that in this application, the displacement change of the target bridge caused by factors such as vehicle speed, load and road conditions is quantified by calculating the standard deviation of all displacements in the displacement trajectory, and the result is used as the reference displacement of the target bridge under the dynamic load in this application. That is, the response of the target bridge under the dynamic load is quantified, thereby determining the displacement (reference displacement) generated when the vehicle passes through the target bridge in the displacement trajectory. In other embodiments, other methods can be used to determine this, which are not limited here.

[0111] In practice, the motion impact response of the load under the selected displacement trajectory can be determined by the following method: First, determine the difference between the reference displacement and the static deflection. Then, divide the result by the static deflection to obtain the motion impact response of the load under the selected displacement trajectory.

[0112] It should be noted that the load analysis described in this application refers to quantifying the deformation and displacement of the bridge under dynamic loads, and then judging the structural bearing capacity of the bridge under dynamic conditions by comparing static deflection and different displacement trajectories. In addition, the motion impact response is a quantitative index that maps the structural state of the target bridge. It reflects the deformation state of the target bridge under different vehicle motion states. When the motion impact response is larger, the degree of deformation of the target bridge under the corresponding vehicle motion state is higher. When the motion impact response is smaller, the degree of deformation of the target bridge under the corresponding vehicle motion state is lower.

[0113] In step 105, the load displacement state of the target bridge under different load conditions is determined based on the abnormal static impact and all motion impact responses, and then a health warning signal corresponding to the different load displacement states is output.

[0114] In some embodiments, reference Figure 3 As shown in the figure, this is a flowchart illustrating the process of determining the load displacement state according to some embodiments of this application. The determination of the load displacement state of the target bridge under different load conditions based on the abnormal static impact and all kinematic impact responses can be achieved through the following steps:

[0115] First, in step 1051, obtain the static deflection of the target bridge;

[0116] Secondly, in 1052, the maximum displacement of the target bridge under static load is determined based on the abnormal static impact and the static deflection.

[0117] Then, in 1053, the maximum displacement of the target bridge under different load conditions is determined based on all motion impact responses and the static deflection.

[0118] Finally, in 1054, the state of load displacement under different load conditions is determined based on the maximum displacement corresponding to each motion impact response and the maximum displacement of the target bridge under static load.

[0119] It should be noted that the maximum displacement of the target bridge under static load refers to the maximum deflection or displacement of the target bridge theoretically calculated under static load, which reflects the degree of deformation of the target bridge under static load conditions. As a preferred embodiment, the maximum displacement of the target bridge under static load based on the abnormal static impact and the static deflection can be achieved in the following way: the result of multiplying the abnormal static impact by 1 and the static deflection is taken as the maximum displacement of the target bridge under static load.

[0120] It should be noted that the maximum displacement of the target bridge under different dynamic loads mentioned in this application refers to the maximum displacement caused by dynamic loads (such as vehicle speed, impact, etc.) when a vehicle passes over the target bridge within different speed ranges. As a preferred embodiment, the maximum displacement of the target bridge under different load conditions can be determined based on all motion impact responses and the static deflection in the following manner: First, select a motion impact response. Then, multiply the result of adding 1 to the motion impact response and the static deflection as the maximum displacement corresponding to the motion impact response. Repeat the above steps to determine the maximum displacement corresponding to the remaining motion impact responses, thereby obtaining the maximum displacement of the target bridge under different load conditions.

[0121] In some embodiments, determining the state of load displacement under different load conditions based on the maximum displacement corresponding to each kinematic impact response and the maximum displacement of the target bridge under static load can be achieved by the following steps:

[0122] The displacement fluctuation of all motion impact responses is determined based on the maximum displacement corresponding to each motion impact response.

[0123] Determine the displacement deviation between the maximum displacement under the static load and the maximum displacement corresponding to each kinematic impact response;

[0124] The state of load displacement under the corresponding load condition is set by the displacement fluctuation and displacement deviation corresponding to each motion impact response.

[0125] In specific implementation, the displacement fluctuation of all motion impact responses can be determined based on the maximum displacement corresponding to each motion impact response in the following way: the standard deviation of the maximum displacement corresponding to all motion impact responses is taken as the displacement fluctuation of all motion impact responses. As a preferred embodiment, the displacement deviation between the maximum displacement under static load and the maximum displacement corresponding to each motion impact response can be determined in the following way: the maximum displacement corresponding to each motion impact response minus the maximum displacement under static load is taken as the displacement deviation between the maximum displacement under static load and the maximum displacement corresponding to each motion impact response.

[0126] It should be noted that the displacement fluctuation amount described in this application characterizes the degree of deviation of the maximum displacement value of the target bridge under different load conditions from the average value. The larger the displacement fluctuation amount, the higher the degree of deviation of the maximum displacement value of the target bridge under different load conditions from the average value; the smaller the displacement fluctuation amount, the lower the degree of deviation of the maximum displacement value of the target bridge under different load conditions from the average value. In addition, the displacement deviation amount characterizes the degree of difference between the maximum displacement of the bridge structure under a predetermined load condition and the maximum displacement under a static load. The larger the displacement deviation amount, the higher the degree of difference between the maximum displacement of the bridge structure under a predetermined load condition and the maximum displacement under a static load; the smaller the displacement deviation amount, the lower the degree of difference between the maximum displacement of the bridge structure under a predetermined load condition and the maximum displacement under a static load.

[0127] In specific implementation, the state of load displacement under the corresponding load conditions for each motion impact response can be set by the displacement fluctuation and displacement deviation corresponding to each motion impact response in the following way: First, select a motion impact response and obtain the displacement deviation corresponding to that motion impact response. Then, add the displacement deviation to the maximum displacement under the static load, and subtract 1.96 times the displacement fluctuation as the upper limit of the interval. Add the displacement deviation to the maximum displacement under the static load, and add 1.96 times the displacement fluctuation as the lower limit of the interval. Thus, the interval is taken as the state of load displacement under the load conditions of that motion impact response. Repeat the above steps to determine the state of load displacement under the load conditions of the remaining motion impact responses. It should be noted that the state of load displacement mentioned in this application refers to the displacement range formed by the structure of the target bridge from the static equilibrium position to the maximum displacement under the dynamic load under the corresponding load conditions. The interval reflects the deformation range of the bridge under the predetermined load and is a key indicator for evaluating the bridge response and safety.

[0128] In addition, it should be noted that by setting the displacement state of the bridge to a normal distribution, in this application, 95% of the data are within ±1.96 standard deviations of the mean. This application determines a confidence interval based on the statistical facts mentioned above, that is: by calculating the displacement deviation and combining it with the displacement fluctuation (standard deviation), the bridge displacement interval is determined, thereby quantifying the health status of the target bridge under different load conditions, that is: the health interval of the target bridge displacement under the corresponding dynamic load.

[0129] In some embodiments, outputting health warning signals corresponding to different states of load displacement can be achieved through the following steps:

[0130] Set the state threshold under each load condition;

[0131] Health warning signals for different states of load displacement are output based on various state thresholds.

[0132] It should be noted that this application sets the displacement distribution in the state of each load displacement to a normal distribution, inputs the state of each load displacement into the normal distribution model, and then selects the displacement corresponding to the 95th quantile in the normal distribution model as the state threshold of the corresponding load displacement state. In other embodiments, other methods can be used to determine this, which are not limited here.

[0133] In specific implementation, the output of health warning signals corresponding to different states of load displacement through various state thresholds can be achieved in the following way: First, monitor the load and actual displacement on the target bridge in real time. Then, obtain the state threshold under the corresponding load condition. When the actual displacement is greater than or equal to the state threshold under the corresponding load condition, send a health warning signal to the health monitoring center of the target bridge. When the actual displacement is less than the state threshold under the corresponding load condition, no action is taken.

[0134] It should be noted that after acquiring real-time images of the target bridge group, this application determines the load displacement state of the target bridge in the real-time images through the above-mentioned process of determining the load displacement state from the status monitoring images. Then, the actual displacement of the bridge is compared with the load displacement state through the state threshold. In addition, when the load displacement state of the target bridge is greater than the preset state threshold, this application can transmit a health warning signal to the health monitoring center of the target bridge through a wireless communication network (such as 4G, 5G or a dedicated wireless network).

[0135] Furthermore, in another aspect of this application, in some embodiments, this application provides a bridge health early warning system based on computer vision, referring to... Figure 4The figure is a schematic diagram of a computer vision-based bridge health early warning system according to some embodiments of this application. The computer vision-based bridge health early warning system 200 includes: a data acquisition module 201, a processing module 202, and an execution module 203, which are described below:

[0136] The acquisition module 201 in this application is mainly used to automatically acquire the status monitoring images of the target bridge, and then determine the actual displacement of the target bridge in each frame of the status monitoring image.

[0137] Processing module 202, in this application, is mainly used to construct a bridge model based on the design information of the target bridge, and then determine the static deflection of the bridge model under preset load conditions;

[0138] In addition, the processing module 202 in this application is also used to determine the abnormal static impact of the target bridge under the preset load by using all actual displacements and the static deflection;

[0139] In addition, the processing module 202 in this application is also used to extract the motion information of vehicles on the target bridge, and then construct the displacement trajectory of the bridge when different vehicle motion states pass through the bridge. The static deflection is used to perform load analysis on various displacement trajectories to obtain the motion impact response of the load under various displacement trajectories.

[0140] The execution module 203 in this application is mainly used to determine the load displacement state of the target bridge under different load conditions based on the abnormal static impact and all motion impact responses, and then output health warning signals corresponding to different load displacement states.

[0141] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described computer vision-based bridge health early warning method.

[0142] In some embodiments, reference Figure 5 This figure is an internal structural diagram of a computer device implementing a computer vision-based bridge health early warning method according to some embodiments of this application. The computer vision-based bridge health early warning method in the above embodiments can... Figure 5 The computer device shown is used to implement this, and the computer device 300 includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.

[0143] The processor 301 may be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more devices used to control the execution of the computer vision-based bridge health warning method in this application.

[0144] The communication bus 302 is used to transmit information between the aforementioned components.

[0145] Memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 303 may exist independently and be connected to processor 301 via communication bus 302. Memory 303 may also be integrated with processor 301.

[0146] The memory 303 stores program code for executing the solution of this application, and its execution is controlled by the processor 301. The processor 301 executes the program code stored in the memory 303. The program code may include one or more software modules. In the above embodiments, the computer vision-based bridge health early warning method can be implemented by the processor 301 and one or more software modules in the program code in the memory 303.

[0147] Communication interface 304 uses any transceiver-like device to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0148] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0149] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device may be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0150] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described computer vision-based bridge health early warning method.

[0151] In summary, the computer vision-based bridge health early warning method and system disclosed in this application automatically acquires state monitoring images of the target bridge to determine the actual displacement of the target bridge in each frame of the state monitoring image; constructs a bridge model based on the design information of the target bridge to determine the static deflection of the bridge model under preset load conditions; determines abnormal static impacts of the target bridge under preset loads by using all actual displacements and the static deflection; extracts the motion information of vehicles on the target bridge to construct the bridge displacement trajectories when different vehicle motion states pass over the bridge; performs load analysis on various displacement trajectories using the static deflection to obtain the motion impact response of the load under various displacement trajectories; determines the load displacement state of the target bridge under different load conditions based on the abnormal static impacts and all motion impact responses, and outputs health early warning signals corresponding to different load displacement states; it can realize continuous monitoring of the bridge, thereby reducing the simulation difficulty of finite element analysis.

[0152] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0153] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A bridge health early warning method based on computer vision, characterized in that, Includes the following steps: Automatically acquire status monitoring images of the target bridge, and then determine the actual displacement of the target bridge in each frame of the status monitoring image; A bridge model is constructed based on the design information of the target bridge, and then the static deflection of the bridge model under preset load conditions is determined. The abnormal static impact of the target bridge under the preset load is determined by all actual displacements and the static deflection. Extract the motion information of vehicles on the target bridge, and then construct the displacement trajectory of the bridge when different vehicle motion states pass through the bridge. Perform load analysis on various displacement trajectories through the static deflection to obtain the motion impact response of the load under various displacement trajectories. Based on the abnormal static impact and all motion impact responses, the load displacement state of the target bridge under different load conditions is determined, and then a health warning signal corresponding to the different load displacement states is output.

2. The method as described in claim 1, characterized in that, Determining the actual displacement of the target bridge in each frame of the status monitoring image specifically includes: Obtain a template image of the target bridge within a specified area; Select one frame of the status monitoring image as the selected status monitoring image; The selected status monitoring image is matched with the template image to obtain a matching image of the selected status monitoring image; Multiple feature point pairs are determined on the matched image; The pixel motion of the selected state monitoring image relative to the template image is determined based on all feature point pairs; Based on the size information of the target bridge, the pixel motion is converted into the actual displacement of the target bridge; Continue to convert the pixel motion of the remaining state monitoring images in the marked area into the actual displacement of the target bridge.

3. The method as described in claim 1, characterized in that, Determining the abnormal static impact of the target bridge under the preset load by using all actual displacements and the static deflection specifically includes: The displacement deviation of each frame of the state monitoring image is determined by all actual displacements and the static deflection. The abnormal static impact of the target bridge under the preset load is determined based on all displacement deviations.

4. The method as described in claim 1, characterized in that, Extracting the motion information of vehicles on the target bridge, and then constructing the displacement trajectory of the bridge when different vehicle motion states pass through the bridge, specifically includes: Acquire vehicle motion information in each frame of the status monitoring image; A dynamic load model of the target bridge is constructed based on the motion information; The dynamic load model is used to determine the displacement trajectory of the bridge when different vehicle motion states pass over the bridge.

5. The method as described in claim 4, characterized in that, Determining the bridge's displacement trajectory when different vehicle motion states pass through the bridge using the dynamic load model specifically includes: Determine all load differences of the target bridge based on the motion information; Determine the motion states of various vehicles on the target bridge by measuring all load differences; The displacement trajectory of the bridge when different vehicle motion states pass through the bridge is determined by the dynamic load model and all vehicle motion states.

6. The method as described in claim 1, characterized in that, By performing load analysis on various displacement trajectories using the static deflection, the specific kinematic impact responses of the loads under various displacement trajectories are obtained, including: Select a displacement trajectory as the selected displacement trajectory; Analyze the reference displacement of the target bridge under the selected displacement trajectory; The motion impact response of the load under the selected displacement trajectory is determined by the reference displacement and the static deflection. Continue to determine the motion impact response of the load under the remaining displacement trajectory.

7. The method as described in claim 1, characterized in that, The determination of the load displacement state of the target bridge under different load conditions based on the abnormal static impact and all kinematic impact responses specifically includes: Obtain the static deflection of the target bridge; The maximum displacement of the target bridge under static load is determined based on the abnormal static impact and the static deflection. The maximum displacement of the target bridge under different load conditions is determined based on all motion impact responses and the static deflection. The state of load displacement under different load conditions is determined based on the maximum displacement corresponding to each motion impact response and the maximum displacement of the target bridge under static load.

8. A bridge health early warning system based on computer vision, characterized in that, include: The acquisition module is used to automatically acquire status monitoring images of the target bridge, and then determine the actual displacement of the target bridge in each frame of the status monitoring image. The processing module is used to construct a bridge model based on the design information of the target bridge, and then determine the static deflection of the bridge model under preset load conditions. The processing module is also used to determine the abnormal static impact of the target bridge under the preset load by using all actual displacements and the static deflection; The processing module is also used to extract the motion information of vehicles on the target bridge, and then construct the displacement trajectory of the bridge when different vehicle motion states pass through the bridge. The static deflection is used to perform load analysis on various displacement trajectories to obtain the motion impact response of the load under various displacement trajectories. The execution module is used to determine the load displacement state of the target bridge under different load conditions based on the abnormal static impact and all motion impact responses, and then outputs health warning signals corresponding to different load displacement states.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the computer vision-based bridge health early warning method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the computer vision-based bridge health early warning method as described in any one of claims 1 to 7.

Images