Bridge health early warning method and system based on computer vision

Abstract

The invention provides a bridge health early warning method and system based on computer vision, and the method comprises the steps: collecting a state monitoring image of a target bridge, and determining the actual displacement of each frame of state monitoring image; constructing a bridge model, and further determining the static deflection of the bridge model under a preset load condition; determining abnormal static impact of the target bridge under the action of a preset load through all the actual displacements and the static deflection; extracting motion information of the vehicle, further constructing displacement tracks of the bridge when the vehicle passes through the bridge in different motion states, and performing load analysis on the various displacement tracks through the static deflection to obtain motion impact responses of loads under the various displacement tracks; and determining load displacement states under different load conditions according to the abnormal static impact and all motion impact responses, and further outputting health early warning signals corresponding to different load displacement states. By the adoption of the scheme, continuous monitoring of the bridge can be achieved, and the simulation difficulty of finite element analysis is lowered.

Description

Technical Field

[0001] This application relates to the technical field of status warning, and in particular, this application relates to a bridge health warning method and system based on computer vision. Background Art

[0002] Status warning is a key technology applied in various industries, especially in the fields of industrial and infrastructure monitoring. It involves using methods such as sensors, data analysis, and machine learning to monitor and evaluate the current state of a system or structure, and predict its future performance and potential risks. The development and application of status warning technology are of great significance for improving the safety, reliability, and efficiency of the system, and it is one of the key technologies for realizing intelligent management and maintenance. With the progress of Internet of Things, big data, and artificial intelligence technologies, status warning technology will continue to develop and provide more accurate and efficient solutions for all walks of life.

[0003] Bridge health warning based on computer vision is to monitor the visual information of a bridge by using image processing and analysis means to evaluate its structural state and safety. Image processing is the core part of this technology. Through steps such as image enhancement, segmentation, and feature extraction, and then analyzing the processed image data to monitor the health state of the bridge in real time, and setting relevant thresholds. Based on the relevant thresholds, potential damages or abnormal changes of the bridge are detected, and then an alarm is issued and a corresponding evaluation report is provided. However, in the existing technology, due to the insufficient application of long-term monitoring technology, most highway bridges are difficult to accumulate continuous displacement data during the operation period, resulting in an increased difficulty in simulating the actual working conditions of the bridge through finite element analysis, and thus unable to provide corresponding warning and alarm values for the deformation monitoring of the bridge structure. Therefore, how to achieve continuous monitoring of the bridge to reduce the simulation difficulty of finite element analysis has become a difficult problem faced by the industry. Summary of the Invention

[0004] This application provides a bridge health warning method and system based on computer vision, which can achieve continuous monitoring of the bridge to reduce the simulation difficulty of finite element analysis.

[0005] In the first aspect, this application provides a bridge health warning method based on computer vision, including the following steps:

[0006] Automatically collect the status monitoring images of the target bridge, and then determine the actual displacement of the target bridge in each frame of the status monitoring image;

[0007] Construct a bridge model based on the design information of the target bridge, and then determine the static deflection of the bridge model under the preset load condition;

[0008] Determine the abnormal static impact of the target bridge under the preset load by using all the actual displacements and the static deflection;

[0009] Extract the motion information of vehicles on the target bridge, and then construct the displacement trajectory of the bridge when different vehicle motion states pass through the bridge. Perform load analysis on various displacement trajectories through the static deflection to obtain the motion impact response of the load under various displacement trajectories;

[0010] Determine the state of the load displacement of the target bridge under different load conditions based on the abnormal static impact and all motion impact responses, and then output health warning signals for different states of the corresponding load displacement.

[0011] In some embodiments, determining the actual displacement of the target bridge in each frame of the state monitoring image specifically includes:

[0012] Obtain the template image of the target bridge in the specified area;

[0013] Select a frame of the state monitoring image as the selected state monitoring image;

[0014] Match the selected state monitoring image with the template image to obtain the matching image of the selected state monitoring image;

[0015] Determine multiple pairs of feature points on the matching image;

[0016] Determine the pixel motion of the selected state monitoring image relative to the template image according to all pairs of feature points;

[0017] Convert the pixel motion into the actual displacement of the target bridge according to the dimension information of the target bridge;

[0018] Continue to convert the pixel motion of the remaining state monitoring images in the marked area into the actual displacement of the target bridge.

[0019] In some embodiments, determining the abnormal static impact of the target bridge under the preset load through all the actual displacements and the static deflection specifically includes:

[0020] Determine the displacement deviation amount of each frame of the state monitoring image through all the actual displacements and the static deflection;

[0021] Determine the abnormal static impact of the target bridge under the preset load according to all the displacement deviation amounts.

[0022] In some embodiments, extracting the motion information of vehicles on the target bridge, and then constructing the displacement trajectory of the bridge when different vehicle motion states pass through the bridge specifically includes:

[0023] Obtain the motion information of the vehicles on each frame of the state monitoring image;

[0024] Construct the dynamic load model of the target bridge according to the motion information;

[0025] Determine the displacement trajectory of the bridge when different vehicle motion states pass through the bridge through the dynamic load model.

[0026] In some embodiments, determining the displacement trajectory of the bridge when different vehicle motion states pass through the bridge through the dynamic load model specifically includes:

[0027] Determine all load difference degrees of the target bridge according to the motion information;

[0028] Determine various vehicle motion states on the target bridge through all load difference degrees;

[0029] Determine the displacement trajectory of the bridge when different vehicle motion states pass through the bridge through the dynamic load model and all vehicle motion states.

[0030] In some embodiments, performing load analysis on various displacement trajectories through the static deflection to obtain the motion impact response of the load under various displacement trajectories specifically includes:

[0031] Select a displacement trajectory as the selected displacement trajectory;

[0032] Analyze the reference displacement of the target bridge under the selected displacement trajectory;

[0033] Determine the motion impact response of the load under the selected displacement trajectory through the reference displacement and the static warping;

[0034] Continue to determine the motion impact response of the load under the remaining displacement trajectories.

[0035] In some embodiments, determining the state of load displacement of the target bridge under different load conditions according to the abnormal static impact and all motion impact responses specifically includes:

[0036] Obtain the static warping of the target bridge;

[0037] Determine the maximum displacement of the target bridge under static load according to the abnormal static impact and the static warping;

[0038] Determine the maximum displacement of the target bridge under different load conditions based on all motion impact responses and the static warping;

[0039] Determine the state of load displacement under different load conditions according to the maximum displacement corresponding to each motion impact response and the maximum displacement of the target bridge under static load.

[0040] In a second aspect, the present application provides a bridge health warning system based on computer vision, including:

[0041] An acquisition module, configured to automatically acquire the state monitoring images of the target bridge, and further determine the actual displacement of the target bridge in each frame of the state monitoring image;

[0042] A processing module, configured to construct a bridge model based on the design information of the target bridge, and further determine the static deflection of the bridge model under preset load conditions;

[0043] The processing module is further configured to determine the abnormal static impact of the target bridge under the action of the preset load through all the actual displacements and the static deflection;

[0044] The processing module is further configured to extract the motion information of the vehicles on the target bridge, and further construct the displacement trajectories of the bridge when different vehicle motion states pass through the bridge, and perform load analysis on various displacement trajectories through the static deflection to obtain the motion impact responses of the loads under various displacement trajectories;

[0045] An execution module, configured to determine the state of the load displacement of the target bridge under different load conditions according to the abnormal static impact and all the motion impact responses, and further output health warning signals for different states corresponding to the load displacement.

[0046] In a third aspect, the present application provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned bridge health warning method based on computer vision are implemented.

[0047] In a fourth aspect, the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned bridge health warning method based on computer vision are implemented.

[0048] The technical solutions provided by the disclosed embodiments of the present application have the following beneficial effects:

[0049] In the bridge health warning method and system based on computer vision provided by the present application, the state monitoring images of the target bridge are automatically acquired, and further the actual displacement of the target bridge in each frame of the state monitoring image is determined; a bridge model is constructed based on the design information of the target bridge, and further the static deflection of the bridge model under preset load conditions is determined; the abnormal static impact of the target bridge under the action of the preset load is determined through all the actual displacements and the static deflection; the motion information of the vehicles on the target bridge is extracted, and further the displacement trajectories of the bridge when different vehicle motion states pass through the bridge are constructed, and load analysis is performed on various displacement trajectories through the static deflection to obtain the motion impact responses of the loads under various displacement trajectories; the state of the load displacement of the target bridge under different load conditions is determined according to the abnormal static impact and all the motion impact responses, and further health warning signals for different states corresponding to the load displacement are output.

[0050] It can be seen that in this application, first, long-term, continuous, and real-time displacement data of the bridge is collected through image monitoring technology, thus avoiding data collection at discrete time points in traditional monitoring technology, providing a richer continuous data source, providing more accurate input conditions for finite element analysis, and based on feature matching for each frame of the status monitoring image, continuous displacement information (actual displacement) in each frame of the status monitoring image is obtained, and then finite element modeling is carried out with the continuous displacement information to obtain a bridge model, thereby reducing the difficulty of simulating the actual working conditions of the bridge through finite element analysis. Secondly, the bridge state (abnormal static impact) of the target bridge under static load and the bridge state (motion impact response) under different load conditions are determined through the bridge model, and the state of the load displacement of the target bridge under different load conditions is determined according to the abnormal static impact and all motion impact responses, so as to quantify the deformation and displacement of the bridge under dynamic load, and then the states of various load displacements are monitored in real time. Furthermore, when a health abnormality occurs, a warning signal is sent to the health monitoring center of the target bridge. In summary, this application can achieve continuous monitoring of the bridge to reduce the simulation difficulty of finite element analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a schematic flow chart of a bridge health warning method based on computer vision according to some embodiments of the present application;

[0052] Figure 2 is a schematic flow chart of determining a displacement trajectory according to some embodiments of the present application;

[0053] Figure 3 is a schematic flow chart of determining the state of load displacement according to some embodiments of the present application;

[0054] Figure 4 is a schematic structural diagram of a bridge health warning system based on computer vision according to some embodiments of the present application;

[0055] Figure 5 is an internal structural diagram of a computer device for implementing the bridge health warning method based on computer vision according to some embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] In order to better understand the technical solutions of the present application, the technical solutions of the present application will be described in detail below in conjunction with the accompanying drawings of the specification and specific embodiments.

[0057] Refer to Figure 1, which is a schematic flowchart of a bridge health warning method based on computer vision according to some embodiments of the present application. The bridge health warning method 100 based on computer vision mainly includes the following steps:

[0058] In step 101, the state monitoring images of the target bridge are automatically collected, and then the actual displacement of the target bridge in each frame of the state monitoring images is determined.

[0059] Preferably, the state monitoring images of the target bridge can be automatically collected by a camera device; it should be noted that the camera device is a high-definition industrial camera, which has a high resolution and can capture the subtle movements and deformations of the bridge, and is used for high-precision displacement and deformation monitoring of the designated areas (such as the mid-span, bearings, crack areas, etc.) of the target bridge. After the video of the marked area is captured by the high-definition industrial camera in the present application, each frame of the image in the video is used as the state monitoring image of the target bridge. In addition, the marked area is the key part of the target bridge where displacement is likely to occur (such as the mid-span, bearings, crack areas, etc.). As a preferred embodiment, measurement point marks can be set at the marked area and used as the target in computer vision technology, so as to provide known reference points in the process of image processing and analysis, help the system more accurately track and measure the displacement changes of the target bridge, and thus improve the accuracy and reliability of the monitoring data.

[0060] In some embodiments, the actual displacement of the target bridge in each frame of the state monitoring image can be achieved by the following steps:

[0061] Obtain the template image of the target bridge in the designated area;

[0062] Select a frame of the state monitoring image as the selected state monitoring image;

[0063] Match the selected state monitoring image with the template image to obtain the matching image of the selected state monitoring image;

[0064] Determine multiple feature point pairs on the matching image;

[0065] Determine the pixel movement of the selected state monitoring image relative to the template image according to all the feature point pairs;

[0066] Convert the pixel movement into the actual displacement of the target bridge according to the size information of the target bridge;

[0067] Continue to convert the pixel movement of the remaining state monitoring images in the marked area into the actual displacement of the target bridge.

[0068] In specific implementation, an image processing software (such as OpenCV) can be used to process the state monitoring image of the target bridge, extract the image of the specified area and optimize it into a template image. In other embodiments, other methods can also be used to determine it, which is not limited here. It should be noted that the template image is a standard image taken under specific conditions and is used as the basis for comparison to achieve feature matching and displacement calculation of the state monitoring image.

[0069] In specific implementation, the selected state monitoring image is matched with the template image to obtain the matching image of the selected state monitoring image, which can be achieved in the following way: First, the position where the coordinates of the upper left corner of the template image coincide with the coordinates of the upper left corner of the selected state monitoring image is used as the initial position of the template image. Subsequently, the width and height of the template image are obtained and used as inputs, and the normalized correlation coefficient matching calculation function in the prior art is used to determine the correlation value between the template image and the selected state monitoring image at the initial position. Again, the template image is continuously slid so that the upper left vertex of the template image traverses each pixel point in the selected state monitoring image, thereby obtaining multiple correlation values. Finally, when the result of the correlation value is 1, the area where the template image is moved to is used as the matching position of the template image in the selected state monitoring image. When the result of the correlation value is not 1, no processing is performed, and the image after the matching is completed is used as the matching image of the selected state monitoring image. In other embodiments, other methods can also be used to implement it, which is not limited here.

[0070] It should be noted that in this application, the matching image refers to the area corresponding to the template image found in the selected state monitoring image, and this area contains parts similar to the features of the template image. In addition, the correlation value characterizes the similarity degree between different areas in the selected state monitoring image and the template image. The larger the correlation value, the higher the similarity degree between the selected area and the template image. The smaller the correlation value, the lower the similarity degree between the selected area and the template image. The value range of the correlation value is (0, 1).

[0071] In specific implementation, determining multiple feature point pairs on the matching image can be achieved in the following manner: First, calculate the second derivative of the matching image to form the Hessian matrix of the matching image. Then, use a Gaussian filter to perform smoothing processing on the image at different scales to generate images at multiple scales, calculate the Hessian matrix for each scale of the image, and thus construct a scale space. Subsequently, after locating the feature points in the matching image through the positions of the extreme points of the Hessian matrix in the scale space, use the Haar wavelet response in the prior art to calculate the main direction of each feature point in the neighborhood area. Again, construct the feature descriptor of each feature point based on the pixel intensity difference within the neighborhood of each feature point. Finally, using the feature descriptors of each feature point as input, use the FLANN algorithm in the prior art to match all feature points, thereby obtaining multiple feature point pairs. In other embodiments, other methods can also be used to determine, which are not limited here.

[0072] It should be noted that the feature points in this application refer to the pixel points with significant features extracted from the matching image. Additionally, the two feature points in the feature point pair are a pixel point pair composed of the feature points recognized from a template image and the feature points corresponding to the template image in the matching image.

[0073] In specific implementation, determining the pixel movement of the selected state monitoring image relative to the template image based on all feature point pairs can be achieved in the following manner: Use the result of dividing the sum of the distances between the corresponding two feature points in all feature point pairs by the total number of feature point pairs in the matching image as the pixel movement of the selected state monitoring image relative to the template image. It should be noted that the distance can be determined by taking the upper left corner of the selected state monitoring image as the origin, taking the horizontal right direction along the origin as the X-axis, and the vertical downward direction as the Y-axis, and then assigning coordinates to each pixel point in the selected image, so as to use the Euclidean distance algorithm in the prior art to determine the pixel distance between the corresponding two feature points in all feature point pairs. In other embodiments, other methods can also be used to determine, which are not limited here.

[0074] It should be noted that the pixel movement refers to the moving distance between the feature points corresponding to the template image in the selected state monitoring image, which reflects the image change caused by bridge deformation or other factors. The greater the pixel movement, the greater the image change caused by bridge deformation or other factors; the smaller the pixel movement, the smaller the image change caused by bridge deformation or other factors.

[0075] In specific implementation, converting the pixel motion into the actual displacement of the target bridge according to the dimension information of the target bridge can be achieved in the following manner, that is: First, obtain the actual length of the target bridge according to the dimension information of the target bridge. Then, obtain the pixel distance between pixel points in the selected state monitoring image. Subsequently, use the value obtained by dividing the actual length by the pixel distance as the conversion coefficient. Finally, multiply the conversion coefficient by the result of the pixel motion corresponding to the selected state monitoring image as the actual displacement of the target bridge. In other embodiments, other methods can also be used to determine it, which is not limited here; it should be noted that the conversion coefficient is the proportional quantity between the pixel distance in the image and the actual length of the target bridge, and is used to convert the pixel motion measured in the image into the actual physical displacement. In addition, the actual displacement refers to the physical displacement observed during the monitoring of the target bridge.

[0076] In step 102, construct a bridge model according to the design information of the target bridge, and then determine the static deflection of the bridge model under the preset load condition.

[0077] In specific implementation, after obtaining the design information of the target bridge, the lateral arch wave of the target bridge can be simplified to a straight beam. Rigid connections are used between the columns above the arch and the main arch and the abdominal arch, and elastic connections are used between the filling above the arch and the main arch. Then, CAD software can be used to create the geometric model of the bridge. The geometric model is imported into a finite element analysis software (such as ANSYS, ABAQUS, SAP2000) and meshed to establish the finite element model of the target bridge, and the finite element model is used as the bridge model in this application. In addition, after determining the bridge model of the target bridge, a preset load can be set on the bridge model to determine the displacement amount occurring in the corresponding marked area of the bridge model, and the displacement amount is used as the static deflection of the corresponding marked area of the bridge model under the preset load. In other embodiments, other methods can also be used to obtain it, which is not limited here.

[0078] In some embodiments, determining the static deflection of the bridge model under the preset load condition can be achieved by the following steps:

[0079] Obtain the preset load of the target bridge;

[0080] Determine the displacement field under the preset load condition through the preset load and the bridge model;

[0081] Generate the static deflection of the bridge model under the preset load condition according to the displacement field.

[0082] In specific implementation, the constant load of the target bridge (including loads continuously acting on the bridge such as the self-weight of the bridge and the weight of permanent facilities) can be used as the preset load in this application. In other embodiments, other methods can also be used to obtain it, which is not limited here.

[0083] It should be noted that after selecting the static analysis (Static Structural Analysis.SSA) module in the finite element analysis software in this application, a displacement field of the bridge model under the preset load condition is generated. By selecting the marked area in the model (usually the mid-span of the bridge, key nodes, or a certain structural part) through the software interface, the displacement result of the marked area is determined, and the vertical displacement of the bridge model in the marked area is used as the static deflection of the bridge model under the preset load condition. In other embodiments, other methods can also be used to determine it, which is not limited here.

[0084] In step 103, an abnormal static impact of the target bridge under the preset load is determined based on all actual displacements and the static deflection.

[0085] In some embodiments, determining an abnormal static impact of the target bridge under the preset load based on all actual displacements and the static deflection can be achieved by the following steps:

[0086] Determine the displacement deviation amount of each frame of the state monitoring image based on all actual displacements and the static deflection;

[0087] Determine an abnormal static impact of the target bridge under the preset load based on all displacement deviation amounts.

[0088] It should be noted that the displacement deviation amount represents the difference between the actual displacement (actual displacement) shown by the target bridge and the expected displacement (static warp) under static load within a specific time frame, characterizing the degree of difference between the actual displacement of the target bridge and its static deflection under static load. The larger the displacement deviation amount, the higher the degree of difference between the actual displacement of the target bridge and its static deflection under static load; the smaller the displacement deviation amount, the lower the degree of difference between the actual displacement of the target bridge and its static deflection under static load. As a preferred embodiment, determining the displacement deviation amount of each frame of the state monitoring image based on all actual displacements and the static deflection can be achieved in the following manner: taking the difference between the actual displacement corresponding to each frame of the state monitoring image and the static deflection as the displacement deviation amount of each frame of the state monitoring image. In other embodiments, other methods can also be used to determine it, which is not limited here.

[0089] In specific implementation, to determine the abnormal static impact of the target bridge under the preset load according to all displacement deviation amounts, the following method can be adopted, that is: First, determine the average value of all displacement deviation amounts, and then, divide the obtained result by the static deflection, and the result is used as the abnormal static impact of the target bridge under the preset load.

[0090] It should be noted that in the context of bridge health monitoring in this application, static impact refers to the static response of the bridge under preset load conditions. This response may include displacements, deformations, etc. caused by the load. The abnormal static impact described in this application is the difference mapping value between the actual structural response and the design expectation of the target bridge under the preset load conditions. The greater the abnormal static impact, the higher the difference between the actual structural response and the design expectation of the target bridge under the preset load conditions; the smaller the abnormal static impact, the lower the difference between the actual structural response and the design expectation of the target bridge under the preset load conditions. In this application, by determining the average value of all displacement deviation amounts and then dividing the obtained result by the static deflection, the result is used as the abnormal static impact of the target bridge under the preset load, that is: quantifying the difference between the actual structural response and the design expectation, so as to provide data support for evaluating the health status of the bridge.

[0091] In step 104, extract the movement information of the vehicle on the target bridge, and then construct the displacement trajectory of the bridge when different vehicle movement states pass through the bridge. Through the static deflection, load analysis is performed on various displacement trajectories to obtain the movement impact response of the load under various displacement trajectories.

[0092] In some embodiments, to extract the movement information of the vehicle on the target bridge and then construct the displacement trajectory of the bridge when different vehicle movement states pass through the bridge, the following method can be adopted:

[0093] Obtain the movement information of the vehicle on each frame of the state monitoring image;

[0094] Construct the dynamic load model of the target bridge according to the movement information;

[0095] Determine the displacement trajectory of the bridge when different vehicle movement states pass through the bridge through the dynamic load model.

[0096] Preferably, this application uses computer vision technology (such as: YOLO, OpenCV, etc.) to process the state monitoring image to identify and extract the vehicles in the state monitoring image, and record the position information (coordinates in the state monitoring image) and time stamp of each vehicle as the movement information of the vehicle on the state monitoring image, so as to obtain the movement information of the vehicle on each frame of the state monitoring image. In other embodiments, other methods can also be used to determine, which are not limited here.

[0097] In specific implementation, the dynamic load model of the target bridge can be constructed according to the movement information in the following manner: First, after obtaining the masses of different vehicles in all the state monitoring images, combining the movement information, input the vehicle information and the movement information into the bridge model, and use the obtained model as the dynamic load model of the target bridge. In other embodiments, other methods can also be used to determine it, which is not limited here.

[0098] Among them, in some embodiments, referring to Figure 2 As shown, this figure is a schematic flowchart of determining the displacement trajectory shown in some embodiments of the present application. The displacement trajectory of the bridge when different vehicle movement states pass through the bridge can be determined through the dynamic load model in the following steps:

[0099] First, in 1041, determine all the load difference degrees of the target bridge according to the movement information;

[0100] Then, in 1042, determine various vehicle movement states on the target bridge through all the load difference degrees;

[0101] Finally, in 1043, determine the displacement trajectory of the bridge when different vehicle movement states pass through the bridge through the dynamic load model and all the vehicle movement states.

[0102] In specific implementation, determining all the load difference degrees of the target bridge according to the movement information can be achieved in the following manner: First, obtain the vehicle position information in the state monitoring images of all adjacent frames according to the movement information. Then, select a pair of adjacent-frame state monitoring images. For each vehicle in this pair of adjacent-frame state monitoring images, extract its vehicle position coordinates from the two-frame state monitoring images, and thus use the Euclidean distance algorithm in the prior art to calculate the displacement of the vehicle between the two-frame state monitoring images. Subsequently, calculate the difference in timestamps between this pair of adjacent-frame state monitoring images, and use the reciprocal of the obtained result multiplied by the displacement of the vehicle between the two-frame state monitoring images as the speed of the vehicle, so as to obtain the speeds of all vehicles in this pair of adjacent-frame state monitoring images. Furthermore, use the average value of the speeds of all vehicles as the vehicle speed of this pair of adjacent-frame state monitoring images. Again, after determining the vehicle speeds between all adjacent-frame state monitoring images, determine the absolute difference between every two vehicle speeds, and use all the absolute differences as the load difference degrees between all vehicles; it should be noted that the load difference degree characterizes the deviation degree of the load of the corresponding vehicle on the target bridge in adjacent state monitoring images. The greater the load difference degree, the higher the deviation degree of the load of the corresponding vehicle on the target bridge in adjacent state monitoring images, and the smaller the load difference degree, the lower the deviation degree of the load of the corresponding vehicle on the target bridge in adjacent state monitoring images.

[0103] It should be noted that based on the fact that the dynamic load effects generated by a vehicle on a bridge vary at different speeds, the load conditions of the target bridge are divided into multiple load modes according to different vehicle speeds, that is: when a vehicle passes through the bridge at different speeds, different impact coefficients and dynamic loads will be generated. The faster the vehicle speed, the stronger the instantaneous impact and dynamic effects on the bridge, which will cause an increase in responses such as bridge displacement, acceleration, and vibration frequency. Conversely, it will cause a decrease in responses such as bridge displacement, acceleration, and vibration frequency. Therefore, as a preferred embodiment, the following method can be used to determine various vehicle motion states on the target bridge through all load differences, that is: after taking all load differences as the similarity index in the K-means clustering algorithm in the prior art, the K-means clustering algorithm in the prior art is used to divide all vehicle speeds into multiple vehicle speed clusters, and then all vehicle speed clusters are respectively used as the vehicle motion states on the target bridge, so as to obtain various vehicle motion states on the target bridge. In other embodiments, other methods can also be used to determine, which are not limited here.

[0104] It should be noted that the displacement trajectory described in this application refers to the displacement, deformation, or movement path of the structure of the target bridge over time under the action of the corresponding load mode. The displacement trajectory usually reflects the change trend of the bridge structure when it is subjected to dynamic loads in a specified area (such as the mid-span, support points, nodes, etc.). The smoother the displacement trajectory, the slower the change trend of the target bridge structure; the steeper the displacement trajectory, the faster the change trend of the target bridge structure. Specifically, when implementing, the following method can be used to determine the displacement trajectory of the bridge when different vehicle motion states pass through the bridge through the dynamic load model and all vehicle motion states, that is: First, select a vehicle motion state, input all vehicle speeds in this vehicle motion state into the dynamic load model, and then, after applying constraint conditions to the dynamic load model to solve the dynamic equation, obtain the displacement amounts of the target bridge at different vehicle speeds in a specified area (key positions of the bridge structure such as the mid-span or support points), and use the set composed of all displacement amounts as the displacement trajectory of the target bridge in this vehicle motion state. Repeat the above steps to determine the displacement trajectories of the target bridge in the remaining vehicle motion states.

[0105] In some embodiments, the following steps can be used to perform load analysis on various displacement trajectories through the static deflection to obtain the motion impact response of the load under various displacement trajectories:

[0106] Select a displacement trajectory as the selected displacement trajectory;

[0107] Analyze the reference displacement of the target bridge under the selected displacement trajectory;

[0108] Determine the motion impact response of the load under the selected displacement trajectory through the reference displacement and the static warping;

[0109] Continue to determine the motion impact response of the load under the remaining displacement trajectory.

[0110] It should be noted that in this application, the standard deviation of all displacement amounts in the displacement trajectory is calculated to quantify the displacement change of the target bridge caused by factors such as vehicle speed, load, and road surface conditions, and the obtained result is used as the reference displacement of the target bridge under the dynamic load in this application, that is: quantifying the response of the target bridge under the dynamic load, so as to determine the displacement amount (reference displacement) generated when the vehicle passes through the target bridge in the displacement trajectory. In other embodiments, other methods can also be used to determine it, which is not limited here.

[0111] When specifically implemented, the motion impact response of the load under the selected displacement trajectory can be determined by the reference displacement and the static camber in the following manner, that is: First, determine the difference between the reference displacement and the static deflection, and then, divide the obtained result by the result of the static deflection as the motion impact response of the load under the selected displacement trajectory.

[0112] It should be noted that the load analysis in this application refers to quantifying the deformation and displacement of the bridge under the dynamic load, and then judging the structural bearing capacity of the bridge under the dynamic condition by comparing the static deflection and different displacement trajectories. In addition, the motion impact response is a quantitative index that maps the structural state of the target bridge, which reflects the deformation state of the target bridge under the action of different vehicle motion states. When the motion impact response is larger, the deformation degree of the target bridge under the action of the corresponding vehicle motion state is higher, and when the motion impact response is smaller, the deformation degree of the target bridge under the action of the corresponding vehicle motion state is lower.

[0113] In step 105, according to the abnormal static impact and all motion impact responses, determine the state of the load displacement of the target bridge under different load conditions, and then output a health warning signal for different states of the corresponding load displacement.

[0114] In some embodiments, refer to Figure 3 As shown, this figure is a schematic flow chart of determining the state of the load displacement shown in some embodiments of this application. The state of the load displacement of the target bridge under different load conditions can be determined according to the abnormal static impact and all motion impact responses by the following steps:

[0115] First, in 1051, obtain the static camber of the target bridge;

[0116] Secondly, in 1052, according to the abnormal static impact and the static camber, determine the maximum displacement of the target bridge under the static load;

[0117] Then, in 1053, determine the maximum displacement of the target bridge under different load conditions based on all the motion impact responses and the static warping;

[0118] Finally, in 1054, determine the state of the load displacement under different load conditions according to the maximum displacement corresponding to each motion impact response and the maximum displacement of the target bridge under static load.

[0119] It should be noted that the maximum displacement of the target bridge under static load refers to the maximum deflection or displacement theoretically calculated for the target bridge under static load, which reflects the degree of deformation of the target bridge under static load conditions; as a preferred embodiment, the maximum displacement of the target bridge under static load can be determined according to the abnormal static impact and the static warping in the following manner, that is: use the result of multiplying the abnormal static impact plus 1 by the static warping as the maximum displacement of the target bridge under static load.

[0120] It should be noted that the maximum displacement of the target bridge under different dynamic loads in this application refers to the maximum displacement caused by dynamic loads (such as vehicle speed, impact, etc.) when the vehicle passes through the target bridge within different driving speed ranges; as a preferred embodiment, the maximum displacement of the target bridge under different load conditions can be determined based on all the motion impact responses and the static warping in the following manner, that is: first, select a motion impact response, then, use the result of multiplying the motion impact response plus 1 by the static warping as the maximum displacement corresponding to the motion impact response, repeat the above steps to determine the maximum displacements corresponding to the remaining motion impact responses, so as to obtain the maximum displacements of the target bridge under different load conditions.

[0121] Among them, in some embodiments, the state of the load displacement under different load conditions can be determined according to the maximum displacement corresponding to each motion impact response and the maximum displacement of the target bridge under static load by the following steps:

[0122] Determine the displacement fluctuation amount of all the motion impact responses according to the maximum displacement corresponding to each motion impact response;

[0123] Determine the displacement deviation amount between the maximum displacement under the static load and the maximum displacement corresponding to each motion impact response;

[0124] Set the state of the load displacement under the load conditions corresponding to each motion impact response through the displacement fluctuation amount corresponding to each motion impact response and the displacement deviation amount corresponding to each motion impact response.

[0125] In specific implementation, to determine the displacement fluctuation amount of all motion shock responses according to the maximum displacement corresponding to each motion shock response, the following method can be adopted, that is: taking the standard deviation of the maximum displacements corresponding to all motion shock responses as the displacement fluctuation amount of all motion shock responses; as a preferred embodiment, to determine the displacement deviation amount between the maximum displacement under the static load and the maximum displacements corresponding to each motion shock response, the following method can be adopted, that is: taking the maximum displacement corresponding to each motion shock response - the maximum displacement under the static load as the displacement deviation amount between the maximum displacement under the static load and the maximum displacement corresponding to each motion shock response.

[0126] It should be noted that in this application, the displacement fluctuation amount characterizes the degree of deviation of the maximum displacement value of the target bridge under different load conditions from the average value. The larger the displacement fluctuation amount, the higher the degree of deviation of the maximum displacement value of the target bridge under different load conditions from the average value; the smaller the displacement fluctuation amount, the lower the degree of deviation of the maximum displacement value of the target bridge under different load conditions from the average value; in addition, the displacement deviation amount characterizes the degree of difference between the maximum displacement of the bridge structure and the maximum displacement under the static load under the predetermined load condition. The larger the displacement deviation amount, the higher the degree of difference between the maximum displacement of the bridge structure and the maximum displacement under the static load under the predetermined load condition; the smaller the displacement deviation amount, the lower the degree of difference between the maximum displacement of the bridge structure and the maximum displacement under the static load under the predetermined load condition.

[0127] In specific implementation, to set the state of the load displacement under the load condition corresponding to each motion shock response through the displacement fluctuation amount corresponding to each motion shock response and the displacement deviation amount corresponding to each motion shock response, the following method can be adopted: that is: first, select a motion shock response and obtain the displacement deviation amount corresponding to this motion shock response. Then, after adding the displacement deviation amount to the maximum displacement under the static load, subtracting 1.96 times the displacement fluctuation amount as the upper limit of the interval, and adding 1.96 times the displacement fluctuation amount to the maximum displacement under the static load as the lower limit of the interval, so as to take this interval as the state of the load displacement under the load condition corresponding to this motion shock response. Repeat the above steps to determine the state of the load displacement under the load condition of the remaining motion shock responses. It should be noted that in this application, the state of the load displacement refers to the displacement amount interval formed by the structure of the target bridge from the static equilibrium position to the maximum displacement under the dynamic load under the corresponding load condition. This interval reflects the deformation range of the bridge under the predetermined load and is a key index for evaluating the bridge response and safety.

[0128] In addition, it should be noted that after setting the displacement state of the bridge to a normal distribution, in the normal distribution, 95% of the data is within ±1.96 standard deviations of the mean. This application determines a confidence interval based on this statistical fact, that is: by calculating the displacement deviation and combining it with the displacement fluctuation (standard deviation), the bridge displacement interval is determined, thereby quantifying the health state of the target bridge under different load conditions, that is: the health interval of the displacement of the target bridge under the corresponding dynamic load.

[0129] In some embodiments, the health warning signals corresponding to different states of the load displacement can be output by the following steps:

[0130] Set the state thresholds for each load condition;

[0131] Output health warning signals corresponding to different states of the load displacement through each state threshold.

[0132] It should be noted that after setting the displacement distribution in the state of each load displacement to a normal distribution, the state of each load displacement is input into the normal distribution model, and then the displacement corresponding to the 95% quantile in the normal distribution model is selected as the state threshold for the state of the corresponding load displacement. In other embodiments, other methods can also be used to determine it, which is not limited here.

[0133] Specifically, the health warning signals corresponding to different states of the load displacement can be output through each state threshold in the following manner, that is: first, the load condition and actual displacement on the target bridge are monitored in real time, and then the state threshold corresponding to the load condition is obtained. When the actual displacement is greater than or equal to the state threshold corresponding to the load condition, a health warning signal is sent to the health monitoring center of the target bridge. When the actual displacement is less than the state threshold corresponding to the load condition, no processing is performed.

[0134] It should be noted that after collecting the real-time images of the target group of bridges, through the above process of determining the state of the load displacement from the state monitoring images, the state of the load displacement of the target bridge in the real-time images is determined. Then, the actual displacement of the bridge in the state of the load displacement is compared with the state threshold. In addition, when the state of the load displacement of the target bridge is greater than the preset state threshold, this application can transmit the health warning signal to the health monitoring center of the target bridge through a wireless communication network (such as 4G, 5G or a dedicated wireless network).

[0135] In addition, on the other hand of this application, in some embodiments, this application provides a bridge health warning system based on computer vision, refer to Figure 4, This figure is a schematic structural diagram of a computer vision-based bridge health warning system according to some embodiments of the present application. The computer vision-based bridge health warning system 200 includes: The acquisition module 201, the processing module 202, and the execution module 203 are described as follows:

[0136] The acquisition module 201. In the present application, the acquisition module 201 is mainly used to automatically acquire the state monitoring images of the target bridge, and then determine the actual displacement of the target bridge in each frame of the state monitoring image;

[0137] The processing module 202. In the present application, the processing module 202 is mainly used to construct a bridge model based on the design information of the target bridge, and then determine the static deflection of the bridge model under the preset load condition;

[0138] In addition, in the present application, the processing module 202 is also used to determine the abnormal static impact of the target bridge under the preset load by all the actual displacements and the static deflection;

[0139] In addition, in the present application, the processing module 202 is also used to extract the motion information of the vehicles on the target bridge, and then construct the displacement trajectories of the bridge when different vehicle motion states pass through the bridge. The load analysis is performed on various displacement trajectories through the static deflection to obtain the motion impact responses of the load under various displacement trajectories;

[0140] The execution module 203. In the present application, the execution module 203 is mainly used to determine the state of the load displacement of the target bridge under different load conditions according to the abnormal static impact and all the motion impact responses, and then output the health warning signals of different states corresponding to the load displacement.

[0141] In addition, the present application also provides a computer device. The computer device includes a memory and a processor. The memory stores code, and the processor is configured to obtain the code and execute the above-mentioned computer vision-based bridge health warning method.

[0142] In some embodiments, refer to Figure 5 , This figure is an internal structure diagram of a computer device for implementing the computer vision-based bridge health warning method according to some embodiments of the present application. The computer vision-based bridge health warning method in the above embodiments can be implemented by Figure 5 the computer device shown. The computer device 300 includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.

[0143] The processor 301 may be a general-purpose central processing unit (CPU), or may be an application-specific integrated circuit (ASIC), or one or more are used to control the execution of the computer vision-based bridge health warning method in this application.

[0144] The communication bus 302 is used to transfer information between the above components.

[0145] The memory 303 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), or other types of dynamic storage devices that can store information and instructions. It may also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disks, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 303 may exist independently and be connected to the processor 301 through the communication bus 302. The memory 303 may also be integrated with the processor 301.

[0146] Among them, the memory 303 is used to store the program code for executing the solution of this application, and is controlled by the processor 301 to execute. The processor 301 is used to execute the program code stored in the memory 303. The program code may include one or more software modules. The computer vision-based bridge health warning method in the above embodiments may be implemented by one or more software modules in the program code in the processor 301 and the memory 303.

[0147] The communication interface 304, using any device such as a transceiver, is used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0148] In a specific implementation, as an example, a computer device may include multiple processors, and each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0149] The above computer device may be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device may be a desktop computer, a laptop computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of the present application do not limit the type of the computer device.

[0150] In addition, the present application also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above bridge health warning method based on computer vision is implemented.

[0151] In summary, in the bridge health warning method and system based on computer vision disclosed in the embodiments of the present application, by automatically collecting the state monitoring images of the target bridge, the actual displacement of the target bridge in each frame of the state monitoring image is determined; a bridge model is constructed based on the design information of the target bridge, and then the static deflection of the bridge model under the preset load condition is determined; the abnormal static impact of the target bridge under the preset load is determined by all the actual displacements and the static deflection; the motion information of the vehicles on the target bridge is extracted, and then the displacement trajectories of the bridge when different vehicle motion states pass through the bridge are constructed, and the load analysis of various displacement trajectories is performed through the static deflection to obtain the motion impact response of the load under various displacement trajectories; according to the abnormal static impact and all the motion impact responses, the state of the load displacement of the target bridge under different load conditions is determined, and then a health warning signal corresponding to different states of the load displacement is output; continuous monitoring of the bridge can be realized to reduce the simulation difficulty of the finite element analysis.

[0152] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present application.

[0153] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and modifications.

Claims

1. A bridge health warning method based on computer vision, characterized in that The steps include the following: Automatically collect the state monitoring images of the target bridge, and then determine the actual displacement of the target bridge in each frame of the state monitoring images; Construct a bridge model based on the design information of the target bridge, and then determine the static deflection of the bridge model under the preset load conditions; Determine the abnormal static impact of the target bridge under the preset load by all the actual displacements and the static deflection; Extract the motion information of the vehicles on the target bridge, and then construct the displacement trajectories of the bridge when different vehicle motion states pass through the bridge. Conduct load analysis on various displacement trajectories through the static deflection to obtain the motion impact responses of the loads under various displacement trajectories; Determine the state of the load displacement of the target bridge under different load conditions according to the abnormal static impact and all the motion impact responses, and then output the health warning signals of different states corresponding to the load displacement.

2. The method according to claim 1, characterized in that Determining the actual displacement of the target bridge in each frame of the state monitoring images specifically includes: Obtain the template image of the target bridge in the specified area; Select a frame of the state monitoring image as the selected state monitoring image; Match the selected state monitoring image with the template image to obtain the matching image of the selected state monitoring image; Determine multiple feature point pairs on the matching image; Determine the pixel motion of the selected state monitoring image relative to the template image according to all the feature point pairs; Convert the pixel motion into the actual displacement of the target bridge according to the dimension information of the target bridge; Continue to convert the pixel motion of the remaining state monitoring images in the marked area into the actual displacement of the target bridge.

3. The method according to claim 1, wherein Determining the abnormal static impact of the target bridge under the preset load by all the actual displacements and the static deflection specifically includes: Determine the displacement deviation amount of each frame of the state monitoring image by all the actual displacements and the static deflection; Determine the abnormal static impact of the target bridge under the preset load according to all the displacement deviation amounts.

4. The method according to claim 1, characterized in that Extracting the motion information of the vehicles on the target bridge, and then constructing the displacement trajectories of the bridge when different vehicle motion states pass through the bridge specifically includes: Obtain the motion information of the vehicles on each frame of the state monitoring image; Construct the dynamic load model of the target bridge according to the motion information; Determine the displacement trajectories of the bridge when different vehicle motion states pass through the bridge through the dynamic load model.

5. The method according to claim 4, wherein Determining the displacement trajectories of the bridge when different vehicle motion states pass through the bridge through the dynamic load model specifically includes: Determine all the load difference degrees of the target bridge according to the motion information; Determine various vehicle motion states on the target bridge through all the load difference degrees; Determine the displacement trajectories of the bridge when different vehicle motion states pass through the bridge through the dynamic load model and all the vehicle motion states.

6. The method according to claim 1, wherein Conduct load analysis on various displacement trajectories through the static deflection to obtain the motion impact responses of the loads under various displacement trajectories specifically includes: Select a displacement trajectory as the selected displacement trajectory; Analyze the reference displacement of the target bridge under the selected displacement trajectory; Determine the motion impact response of the load under the selected displacement trajectory through the reference displacement and the static deflection; Continue to determine the motion impact responses of the loads under the remaining displacement trajectories.

7. The method according to claim 1, characterized in that, Determining the state of load displacement of the target bridge under different load conditions based on the abnormal static impact and all motion impact responses specifically includes: Obtaining the static camber of the target bridge; Determining the maximum displacement of the target bridge under static load according to the abnormal static impact and the static camber; Determining the maximum displacement of the target bridge under different load conditions based on all motion impact responses and the static camber; Determining the state of load displacement under different load conditions according to the maximum displacement corresponding to each motion impact response and the maximum displacement of the target bridge under static load.

8. A bridge health warning system based on computer vision, characterized in that, Including: A collection module, configured to automatically collect the state monitoring images of the target bridge, and further determine the actual displacement of the target bridge in each frame of the state monitoring images; A processing module, configured to construct a bridge model based on the design information of the target bridge, and further determine the static deflection of the bridge model under preset load conditions; The processing module is further configured to determine the abnormal static impact of the target bridge under the action of the preset load through all actual displacements and the static deflection; The processing module is further configured to extract the motion information of the vehicles on the target bridge, and further construct the displacement trajectories of the bridge when different vehicle motion states pass through the bridge, and perform load analysis on various displacement trajectories through the static deflection to obtain the motion impact responses of the loads under various displacement trajectories; An execution module, configured to determine the state of load displacement of the target bridge under different load conditions according to the abnormal static impact and all motion impact responses, and further output health warning signals of different states corresponding to the load displacement.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the computer vision-based bridge health warning method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the steps of the computer vision-based bridge health warning method according to any one of claims 1 to 7 are implemented.

Images