Method and system for high-precision acquisition of bridge vibration based on computer vision

Abstract

The invention relates to the technical field of image processing, and discloses a method and system for high-precision acquisition of bridge vibration based on computer vision, and the method comprises the steps: determining a to-be-detected bridge, recording a vibration video of the to-be-detected bridge, carrying out the frame-by-frame extraction of the vibration video, and obtaining the image data of a plurality of to-be-detected bridges; performing gray processing on the image data to obtain gray image data of the to-be-detected bridge; performing edge detection on the grayscale image data by using a Canny operator to obtain a plurality of edge points of the to-be-detected bridge, and calculating the vertical displacement of each edge point one by one; calculating the sub-pixel precision of the edge displacement by using a Zernike moment, and obtaining the actual size data of the to-be-detected bridge; and calculating a scale factor corresponding to each piece of image data according to the sub-pixel precision and the actual size data, and calculating the actual vibration displacement of the to-be-detected bridge according to the scale factors. The defects of a traditional displacement sensor are overcome, and efficient, accurate and high-precision bridge displacement detection is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for obtaining bridge vibration with high precision based on computer vision. Background Art

[0002] As one of the key indicators of bridge health monitoring, bridge vibration data must be collected efficiently and accurately without interfering with traffic flow, so as to timely identify changes in dynamic characteristics and potential damage, thereby preventing potential safety hazards from causing major safety accidents. Therefore, it is crucial to regularly test the dynamic performance of bridges. At present, most detection technologies rely on contact sensors, such as accelerometers, to collect bridge vibration data. These sensors must be installed on the bridge structure, and each sensor can only monitor the vibration of a single location, which requires the use of a data acquisition system, resulting in a time-consuming, labor-intensive, and costly detection process with limited accuracy. In addition, computer vision-based vibration detection methods, such as template matching technology, although they do not rely on contact sensors, require the installation of artificial targets at the monitoring points, and this method is computationally intensive, time-consuming, and labor-intensive.

[0003] Therefore, it is necessary to design a method and system for obtaining bridge vibration with high precision based on computer vision to solve the problems existing in current technology. Summary of the invention

[0004] In view of this, the present invention proposes a method and system for acquiring bridge vibration with high precision based on computer vision, aiming to solve the problems of time-consuming, labor-intensive, high cost and limited accuracy in bridge vibration monitoring in current technology.

[0005] In one aspect, the present invention provides a method for obtaining bridge vibration with high precision based on computer vision, comprising the following steps:

[0006] S100: determining a bridge to be detected, recording a vibration video of the bridge to be detected, and extracting the vibration video frame by frame to obtain image data of a plurality of bridges to be detected;

[0007] S200: grayscale processing is performed on the image data to obtain grayscale image data of the bridge to be detected:

[0008] S300: performing edge detection on the grayscale image data using a Canny operator to obtain a plurality of edge points of the bridge to be detected, and calculating the vertical displacement of each edge point one by one;

[0009] S400: Calculate the sub-pixel accuracy of edge displacement using Zernike moments, and obtain actual size data of the bridge to be detected; calculate a proportional factor corresponding to each image data according to the sub-pixel accuracy and the actual size data, and calculate the actual vibration displacement of the bridge to be detected according to the proportional factor.

[0010] Further, determining a bridge to be detected, recording a vibration video of the bridge to be detected, and extracting the vibration video frame by frame to obtain image data of several bridges to be detected includes:

[0011] Determine the monitoring point information of the bridge to be detected, and set up a high-definition camera at a first distance from the bridge to be detected based on the monitoring point information; wherein the angle between the optical axis direction of the high-definition camera and the horizontal line ranges from 0° to 45°;

[0012] When a vehicle passes by the bridge to be inspected or a gust of wind excites the bridge to be inspected, recording a vibration video of the bridge to be inspected;

[0013] The vibration video is extracted frame by frame to obtain a plurality of image data, and the image data are numbered in time series.

[0014] Furthermore, grayscale processing is performed on the image data to obtain grayscale image data of the bridge to be detected, including:

[0015] Performing grayscale processing on the image data by using a weighted average grayscale processing method to obtain a grayscale value of each pixel in the image data, and constructing a grayscale image matrix according to the grayscale value;

[0016] The gray value is obtained by the following formula:

[0017] F(i,j)=0.30R(i,j)+0.59G(i,j)+0.11B(i,j);

[0018] Among them, F represents the grayscale value; R represents the value of red in the image data; G represents the value of blue in the image data; B represents the value of green in the image data, and (i, j) represents the coordinates of the pixel point in the image data.

[0019] Furthermore, the Canny operator is used to perform edge detection on the grayscale image data to obtain a plurality of edge points of the bridge to be detected, and the vertical displacement of each edge point is calculated one by one, including:

[0020] Gaussian filtering, pixel gradient calculation, non-maximum pixel gradient suppression;

[0021] The Gaussian filtering includes:

[0022] The grayscale image data is Gaussian filtered using a Gaussian kernel, and the grayscale image data is discretized into a matrix form to obtain a Gaussian kernel matrix; wherein the Gaussian kernel form is:

[0023]

[0024] The Gaussian kernel matrix is:

[0025]

[0026] The pixel gradient calculation includes:

[0027] Calculate the pixel gradient matrix Gx in the x direction of the grayscale image data and the pixel gradient matrix Gy in the y direction of the grayscale image data based on the Sobel operator;

[0028] The specific form of the pixel gradient matrix Gx in the x direction is:

[0029]

[0030] The specific form of the pixel gradient matrix Gy in the y direction is:

[0031]

[0032] Where I represents the grayscale image matrix;

[0033] The non-maximum pixel gradient suppression includes:

[0034] Calculate the gradient strength of the pixel according to the pixel gradient matrix Gx in the x-direction and the pixel gradient matrix Gy in the y-direction;

[0035] The gradient strength of the current pixel is compared with the gradient strength of adjacent pixels along the positive and negative gradient directions, and it is determined whether to retain the current pixel as an edge point according to the comparison result.

[0036] Further, judging whether to retain the current pixel as an edge point according to the comparison result includes:

[0037] If the gradient intensity of the current pixel is the maximum value, retaining the current pixel as an edge point;

[0038] If the gradient intensity of the current pixel is not the maximum value, the current pixel is suppressed and is not regarded as an edge point.

[0039] Furthermore, the method further comprises: performing edge detection on the grayscale image data using a Canny operator to obtain a plurality of edge points of the bridge to be detected, and calculating the vertical displacement of each edge point one by one;

[0040] The distance between each edge point and the image frame is calculated one by one, and the distance is recorded as a vertical displacement; wherein the unit of the vertical displacement is a pixel.

[0041] Furthermore, when calculating the sub-pixel accuracy of edge displacement using Zernike moments, it includes:

[0042] Calculating the parameters of the edge point based on the Zernike moment, the parameters include background gray value h, step height k, vertical distance l from the center of the disk to the edge, and angle φ between the vertical line and the x-axis;

[0043] By calculating three Zernike moments, namely Z'00, Z'11, and Z'20, the background gray value h, step height k, vertical distance l, and angle φ are obtained;

[0044] The angle φ is obtained by the following formula:

[0045]

[0046]

[0047]

[0048] The vertical distance l is obtained by the following formula:

[0049]

[0050] The step height k is obtained by the following formula:

[0051]

[0052] Furthermore, when calculating the sub-pixel accuracy of edge displacement using Zernike moments, it also includes:

[0053] Determine the Zernike moment Zn,m before the grayscale image data is rotated and the Zernike moment Z'n,m after the grayscale image data is rotated by an angle φ; wherein the relationship is as shown in the following formula:

[0054] Z′ n，m =Z n，m exp(-jφ);

[0055] Let f(x,y) be the rotated image, then:

[0056]

[0057] The edge point coordinates (asc, bsc) with sub-pixel accuracy are calculated from the edge point coordinates (a, b):

[0058]

[0059] Further, calculating a proportional factor corresponding to each of the image data according to the sub-pixel accuracy and the actual size data, and calculating the actual vibration displacement of the bridge to be detected according to the proportional factor, includes:

[0060] The actual size data is the actual physical length D of the bridge to be inspected;

[0061] The proportional factor is calculated according to the actual physical length D and the pixel length P of the grayscale image data; wherein the proportional factor is obtained by the following formula:

[0062]

[0063] Calculate the edge sub-pixel precision displacement according to the edge point coordinates (asc, bsc) with sub-pixel precision;

[0064] The edge sub-pixel precision displacement is multiplied by a proportional factor to obtain the actual vibration displacement of the bridge to be detected.

[0065] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention performs grayscale processing and edge detection on each frame of the bridge vibration video, selects the edge at the displacement monitoring point required, uses the programmed edge tracking program to track the vertical displacement of the edge in each frame of the image, and uses the advantages of the Zernike moment to calculate the sub-pixel precision displacement of the edge through the programmed program for calculating the sub-pixel precision of the edge, and calculates the proportional factor by tracking the structural components of the known actual size on the bridge. Finally, this method can calculate the high-precision bridge vibration displacement, overcomes the shortcomings of traditional displacement sensors, and achieves efficient, accurate and high-precision bridge displacement detection.

[0066] In another aspect, the present invention also proposes a system for obtaining bridge vibration with high precision based on computer vision, comprising:

[0067] An extraction module is configured to determine a bridge to be detected, record a vibration video of the bridge to be detected, and extract the vibration video frame by frame to obtain image data of a plurality of bridges to be detected;

[0068] A processing module is configured to perform grayscale processing on the image data to obtain grayscale image data of the bridge to be detected;

[0069] An edge detection module is configured to perform edge detection on the grayscale image data using a Canny operator to obtain a plurality of edge points of the bridge to be detected, and calculate the vertical displacement of each edge point one by one;

[0070] The sub-pixel precision calculation module is configured to calculate the sub-pixel precision of edge displacement using Zernike moments and obtain actual size data of the bridge to be detected; calculate the proportional factor corresponding to each image data according to the sub-pixel precision and the actual size data, and calculate the actual vibration displacement of the bridge to be detected according to the proportional factor.

[0071] It can be understood that the above-mentioned method and system for obtaining bridge vibration with high precision based on computer vision have the same beneficial effects, which will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0073] Figure 1 A flowchart of a method for obtaining bridge vibration with high precision based on computer vision provided by an embodiment of the present invention;

[0074] Figure 2 A structural block diagram of a system for obtaining bridge vibration with high precision based on computer vision provided by an embodiment of the present invention;

[0075] Figure 3 A schematic diagram of the background gray value h, step height k, vertical distance l and angle φ of the transverse steel pipe member of the bridge provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0076] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to be able to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0077] See also Figure 1 As shown, in some embodiments of the present application, this embodiment provides a method for obtaining bridge vibration with high precision based on computer vision, comprising the following steps:

[0078] S100: determining a bridge to be detected, recording a vibration video of the bridge to be detected, and extracting the vibration video frame by frame to obtain image data of a plurality of bridges to be detected;

[0079] S200: performing grayscale processing on the image data to obtain grayscale image data of the bridge to be detected;

[0080] S300: performing edge detection on the grayscale image data using a Canny operator to obtain a plurality of edge points of the bridge to be detected, and calculating the vertical displacement of each edge point one by one;

[0081] S400: Calculate the sub-pixel accuracy of edge displacement using Zernike moments, and obtain actual size data of the bridge to be detected; calculate a proportional factor corresponding to each image data according to the sub-pixel accuracy and the actual size data, and calculate the actual vibration displacement of the bridge to be detected according to the proportional factor.

[0082] It can be seen that the method for obtaining bridge vibration with high precision based on computer vision provided in this embodiment can realize real-time monitoring of the health status of the bridge structure by analyzing the vibration of the bridge frame by frame. This method not only improves the accuracy of monitoring, but also greatly reduces the need for manual intervention and reduces the monitoring cost due to the use of computer vision technology. In addition, this method also has good adaptability and can be applied to different types of bridge structures, including but not limited to beam bridges, arch bridges, suspension bridges, etc.

[0083] Specifically, determining a bridge to be detected, recording a vibration video of the bridge to be detected, and extracting the vibration video frame by frame to obtain image data of several bridges to be detected includes:

[0084] Determine the monitoring point information of the bridge to be detected, and set up a high-definition camera at a first distance from the bridge to be detected based on the monitoring point information; wherein the angle between the optical axis direction of the high-definition camera and the horizontal line ranges from 0° to 45°;

[0085] When a vehicle passes by the bridge to be inspected or a gust of wind excites the bridge to be inspected, recording a vibration video of the bridge to be inspected;

[0086] The vibration video is extracted frame by frame to obtain a plurality of image data, and the image data are numbered in time series.

[0087] It is understandable that the first distance is as close to the bridge as possible, because the closer the HD camera is to the bridge, the higher the accuracy of the result, but it is necessary to ensure that the angle between the camera's line of sight and the horizontal line does not exceed 45 degrees, keep the monitoring point within the camera image, and record the vibration video of the bridge when there are vehicles passing by the bridge or gusts of wind. Use the program to decompose the recorded video into frames of images. The more frames per second the camera has, the more images are obtained, and finally the more vibration points are obtained, providing more data for the later calculation of parameters such as the natural frequency.

[0088] Specifically, performing grayscale processing on the image data to obtain the grayscale image data of the bridge to be detected includes:

[0089] Performing grayscale processing on the image data by using a weighted average grayscale processing method to obtain a grayscale value of each pixel in the image data, and constructing a grayscale image matrix according to the grayscale value;

[0090] The gray value is obtained by the following formula:

[0091] F(i,j)=0.30R(i,j)+0.59G(i,j)+0.11B(i,j);

[0092] Among them, F represents the grayscale value; R represents the value of red in the image data; G represents the value of blue in the image data; B represents the value of green in the image data, and (i, j) represents the coordinates of the pixel point in the image data.

[0093] It can be understood that the acquired image data is grayscaled. Image grayscale refers to an image with only one sampled color per pixel. Such images are usually displayed as grayscale from the darkest black to the brightest white. Grayscale means no color, and all RGB color components are equal.

[0094] Specifically, the Canny operator is used to perform edge detection on the grayscale image data to obtain a plurality of edge points of the bridge to be detected, and the vertical displacement of each edge point is calculated one by one, including:

[0095] Gaussian filtering, pixel gradient calculation, non-maximum pixel gradient suppression;

[0096] The Gaussian filtering includes:

[0097] The grayscale image data is Gaussian filtered using a Gaussian kernel, and the grayscale image data is discretized into a matrix form to obtain a Gaussian kernel matrix; wherein the Gaussian kernel form is:

[0098]

[0099] The Gaussian kernel matrix is:

[0100]

[0101] The pixel gradient calculation includes:

[0102] Calculate the pixel gradient matrix Gx in the x direction of the grayscale image data and the pixel gradient matrix Gy in the y direction of the grayscale image data based on the Sobel operator;

[0103] The specific form of the pixel gradient matrix Gx in the x direction is:

[0104]

[0105] The specific form of the pixel gradient matrix Gy in the y direction is:

[0106]

[0107] Where I represents the grayscale image matrix;

[0108] The non-maximum pixel gradient suppression includes:

[0109] Calculate the gradient strength of the pixel according to the pixel gradient matrix Gx in the x-direction and the pixel gradient matrix Gy in the y-direction;

[0110] The gradient strength of the current pixel is compared with the gradient strength of adjacent pixels along the positive and negative gradient directions, and it is determined whether to retain the current pixel as an edge point according to the comparison result.

[0111] In this embodiment, the gradient intensity refers to the degree of change in brightness of a pixel point, which reflects the clarity of the edge in the image.

[0112] It is understandable that Gaussian filtering is a smoothing technique used to reduce image noise and details for better edge detection. In this embodiment, the construction of the Gaussian kernel matrix is ​​based on the Gaussian distribution function, and its core is to perform a convolution operation on the image through a two-dimensional Gaussian function to achieve smoothing of the image. The size and standard deviation of the Gaussian kernel matrix have a direct impact on the filtering effect and need to be adjusted according to the characteristics and noise level of the actual bridge image.

[0113] Pixel gradient calculation is a key step in edge detection, which involves calculating the gradient magnitude and direction of each pixel in the image. In this embodiment, by calculating the horizontal and vertical gradients of each pixel, the gradient magnitude and direction of the point can be obtained. These gradient information is the basis for subsequent edge point detection.

[0114] Non-maximum pixel gradient suppression is a refinement step in edge detection, which aims to suppress those pixels that are not edges and retain only those pixels with larger gradient amplitude as potential edge points. In this embodiment, by comparing the gradient amplitude of each pixel with its neighboring pixels, the edge information can be effectively highlighted while removing the interference of non-edge areas.

[0115] Through the above steps, a series of edge points can be obtained, which represent the contour changes of the bridge during vibration. Subsequently, the vertical displacement of each edge point is calculated one by one to provide basic data for subsequent vibration displacement calculation. The whole process reflects the high accuracy and efficiency of this method in bridge vibration monitoring, and provides strong technical support for bridge health monitoring and maintenance.

[0116] Specifically, judging whether to retain the current pixel as an edge point according to the comparison result includes:

[0117] If the gradient intensity of the current pixel is the maximum value, retaining the current pixel as an edge point;

[0118] If the gradient intensity of the current pixel is not the maximum value, the current pixel is suppressed and is not regarded as an edge point.

[0119] It is understandable that in the edge detection process, the maximum value of the gradient intensity usually corresponds to the most obvious part of the edge in the image. By comparing the gradient intensity of the pixel point with the gradient intensity of the neighboring pixel points, the edge point can be effectively identified. If the gradient intensity of a pixel point is the maximum value in its neighborhood, then this point is likely to be on the edge of the bridge outline and should be retained as an edge point. On the contrary, if the gradient intensity of a pixel point is not the maximum value, then it may be located in a non-edge area. At this time, the pixel point should be suppressed and not regarded as an edge point. In this way, the edge contour of the bridge when it vibrates can be effectively extracted from the image, providing accurate data support for subsequent displacement calculations. This edge point screening method based on gradient intensity not only improves the accuracy of edge detection, but also helps to reduce the interference of noise and non-edge information, thereby ensuring the high accuracy and reliability of bridge vibration monitoring.

[0120] Specifically, the Canny operator is used to perform edge detection on the grayscale image data to obtain a plurality of edge points of the bridge to be detected, and the vertical displacement of each edge point is calculated one by one, and the method further includes:

[0121] The distance between each edge point and the image frame is calculated one by one, and the distance is recorded as a vertical displacement; wherein the unit of the vertical displacement is a pixel.

[0122] It is understandable that the calculation of vertical displacement is a key step in bridge vibration analysis, which can reflect the actual displacement of the bridge during vibration. In this embodiment, the vertical displacement of each edge point can be obtained by calculating the distance from each edge point to the edge of the image frame. Since the image data has been converted into a grayscale image, the calculated displacement unit is pixel. This pixel-based displacement calculation method is simple and effective, and can provide intuitive data support for the health status assessment of the bridge structure.

[0123] Specifically, when using Zernike moments to calculate the sub-pixel accuracy of edge displacement, it includes:

[0124] Calculating the parameters of the edge point based on the Zernike moment, the parameters include background gray value h, step height k, vertical distance l from the center of the disk to the edge, and angle φ between the vertical line and the x-axis;

[0125] By calculating three Zernike moments, namely Z'00, Z'11, and Z'20, the background gray value h, step height k, vertical distance l, and angle φ are obtained;

[0126] The angle φ is obtained by the following formula:

[0127]

[0128]

[0129]

[0130] The vertical distance l is obtained by the following formula:

[0131]

[0132] The step height k is obtained by the following formula:

[0133]

[0134] It is understood that the Zernike moment is a mathematical tool for image analysis, which can provide a description of the shape of the image and is invariant to the rotation and scale changes of the image. In this embodiment, the sub-pixel accuracy of the edge displacement is calculated using the Zernike moment to further improve the accuracy of the bridge vibration displacement measurement. The calculated background gray value h, step height k, vertical distance l from the center of the disk to the edge, and the angle φ between the vertical line and the x-axis can more accurately describe the position of the edge point, thereby obtaining a higher displacement measurement accuracy than the traditional pixel level.

[0135] Specifically, by calculating three Zernike moments, namely Z'00, Z'11, and Z'20, the background gray value h, step height k, vertical distance l, and angle φ are obtained, which is based on the definition and properties of Zernike moments. Z'00 reflects the background brightness level of the image, while Z'11 and Z'20 are related to the shape and position of the edge. Through the calculation of these moments, the precise position information of the edge point can be obtained, and then the sub-pixel displacement of the bridge vibration can be calculated.

[0136] Table 1 shows the complex numbers corresponding to Z'00, Z'11, and Z'20;

[0137] Table 1

[0138] m / n 0 1 0 1 / 1 / X-yj 2 <h2 style=";text-align:left;direction:ltr"><![CDATA[2x <h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +2y<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> -1]]><h2 style=";text-align:left;direction:ltr"> / 3 / <h2 style=";text-align:left;direction:ltr"><![CDATA[(3x <h2 style=";text-align:left;direction:ltr"> 3 <h2 style=";text-align:left;direction:ltr"> +3xy<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> -2x)+(3y<h2 style=";text-align:left;direction:ltr"> 3 <h2 style=";text-align:left;direction:ltr"> +3xy-2y)j]]><h2 style=";text-align:left;direction:ltr"> 4 <![CDATA[6x 4 +6y 4 +12x 2 y]]> /

[0139] Specifically, when using Zernike moments to calculate the sub-pixel accuracy of edge displacement, it also includes:

[0140] Determine the Zernike moment Zn,m before the grayscale image data is rotated and the Zernike moment Z'n,m after the grayscale image data is rotated by an angle φ; wherein the relationship is as shown in the following formula:

[0141] Z′ n,m =Z n,m exp(-jφ);(3x 3 +3xy 2 -2x)+(3y 3 +3x y -2y)j

[0142] Let f(x,y) be the rotated image, then:

[0143]

[0144] The edge point coordinates (asc, bsc) with sub-pixel accuracy are calculated from the edge point coordinates (a, b):

[0145]

[0146] It can be understood that after determining the relationship between the Zernike moments before and after the rotation, the coordinates of the edge points of the rotated image can be obtained by calculating the rotation angle φ. This process involves comparing the Zernike moments before the rotation with the Zernike moments after the rotation, and obtaining the rotation angle φ through mathematical transformation. Then, using the rotation angle φ and the original coordinates (a, b) of the edge point, the coordinates (asc, bsc) of the edge point with sub-pixel accuracy can be calculated. This calculation process is achieved through a series of mathematical formulas and transformations, which ensures the accuracy of the edge point position, thereby providing a basis for the accurate measurement of the vibration displacement of the bridge. Through the above steps, the edge point coordinates with sub-pixel accuracy can be obtained, which reflect the precise displacement of the bridge during vibration.

[0147] Specifically, calculating the proportionality factor corresponding to each of the image data according to the sub-pixel accuracy and the actual size data, and calculating the actual vibration displacement of the bridge to be detected according to the proportionality factor, includes:

[0148] The actual size data is the actual physical length D of the bridge to be inspected;

[0149] The proportional factor is calculated according to the actual physical length D and the pixel length P of the grayscale image data; wherein the proportional factor is obtained by the following formula:

[0150]

[0151] Calculate the edge sub-pixel precision displacement according to the edge point coordinates (asc, bsc) with sub-pixel precision;

[0152] The edge sub-pixel precision displacement is multiplied by a proportional factor to obtain the actual vibration displacement of the bridge to be detected.

[0153] In this embodiment, sub-pixel precision displacement refers to the displacement of the bridge during vibration, and its measurement accuracy exceeds the limitation of traditional pixel level.

[0154] It can be understood that the scale factor corresponding to each image data is calculated based on the sub-pixel accuracy of each edge point and the actual size data of the bridge. The calculation of the scale factor is based on the proportional relationship between the actual size of the bridge and the size of the bridge in the image, which can convert the displacement in the image into the actual physical displacement. In this way, the actual vibration displacement of the bridge can be obtained, providing accurate data support for the health monitoring and maintenance of the bridge. This method not only improves the accuracy of bridge vibration monitoring, but also greatly reduces the need for manual intervention and reduces the monitoring cost due to the use of computer vision technology. In addition, the method has good adaptability and can be applied to different types of bridge structures, including but not limited to beam bridges, arch bridges, suspension bridges, etc.

[0155] See also Figure 2 As shown, in some embodiments of the present application, this embodiment provides a system for obtaining bridge vibration with high precision based on computer vision, including:

[0156] An extraction module is configured to determine a bridge to be detected, record a vibration video of the bridge to be detected, and extract the vibration video frame by frame to obtain image data of a plurality of bridges to be detected;

[0157] A processing module is configured to perform grayscale processing on the image data to obtain grayscale image data of the bridge to be detected;

[0158] An edge detection module is configured to perform edge detection on the grayscale image data using a Canny operator to obtain a plurality of edge points of the bridge to be detected, and calculate the vertical displacement of each edge point one by one;

[0159] The sub-pixel precision calculation module is configured to calculate the sub-pixel precision of edge displacement using Zernike moments and obtain actual size data of the bridge to be detected; calculate the proportional factor corresponding to each image data according to the sub-pixel precision and the actual size data, and calculate the actual vibration displacement of the bridge to be detected according to the proportional factor.

[0160] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0161] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0162] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for obtaining bridge vibration with high precision based on computer vision, characterized in that: include: Determine a bridge to be detected, record a vibration video of the bridge to be detected, and extract the vibration video frame by frame to obtain image data of several bridges to be detected; Performing grayscale processing on the image data to obtain grayscale image data of the bridge to be detected; Perform edge detection on the grayscale image data using the Canny operator to obtain a number of edge points of the bridge to be detected, and calculate the vertical displacement of each edge point one by one; Calculate the sub-pixel accuracy of edge displacement using Zernike moments, and obtain actual size data of the bridge to be inspected; The proportional factor corresponding to each image data is calculated according to the sub-pixel accuracy and the actual size data, and the actual vibration displacement of the bridge to be detected is calculated according to the proportional factor.

2. The method for obtaining bridge vibration with high precision based on computer vision according to claim 1 is characterized in that: Determining a bridge to be detected, recording a vibration video of the bridge to be detected, and extracting the vibration video frame by frame to obtain image data of a plurality of bridges to be detected, including: Determine the monitoring point information of the bridge to be detected, and set up a high-definition camera at a first distance from the bridge to be detected based on the monitoring point information; wherein the angle between the optical axis direction of the high-definition camera and the horizontal line ranges from 0° to 45°; When a vehicle passes by the bridge to be inspected or a gust of wind excites the bridge to be inspected, recording a vibration video of the bridge to be inspected; The vibration video is extracted frame by frame to obtain a plurality of image data, and the image data are numbered in time series.

3. The method for obtaining bridge vibration with high precision based on computer vision according to claim 2 is characterized in that: Performing grayscale processing on the image data to obtain grayscale image data of the bridge to be detected includes: Performing grayscale processing on the image data by using a weighted average grayscale processing method to obtain a grayscale value of each pixel in the image data, and constructing a grayscale image matrix according to the grayscale value; The gray value is obtained by the following formula: F(i,j)=0.30R(i,j)+0.59G(i,j)+0.11B(i,j); Among them, F represents the grayscale value; R represents the value of red in the image data; G represents the value of blue in the image data; B represents the value of green in the image data, and (i, j) represents the coordinates of the pixel point in the image data.

4. The method for obtaining bridge vibration with high precision based on computer vision according to claim 1 is characterized in that: Using the Canny operator to perform edge detection on the grayscale image data to obtain a plurality of edge points of the bridge to be detected, and calculating the vertical displacement of each edge point one by one, includes: Gaussian filtering, pixel gradient calculation, non-maximum pixel gradient suppression; The Gaussian filtering includes: The grayscale image data is Gaussian filtered using a Gaussian kernel, and the grayscale image data is discretized into a matrix form to obtain a Gaussian kernel matrix; wherein the Gaussian kernel form is: The Gaussian kernel matrix is: The pixel gradient calculation includes: Calculate the pixel gradient matrix Gx in the x direction of the grayscale image data and the pixel gradient matrix Gy in the y direction of the grayscale image data based on the Sobel operator; The specific form of the pixel gradient matrix Gx in the x direction is: The specific form of the pixel gradient matrix Gy in the y direction is: Where I represents the grayscale image matrix; The non-maximum pixel gradient suppression includes: Calculate the gradient strength of the pixel according to the pixel gradient matrix Gx in the x-direction and the pixel gradient matrix Gy in the y-direction; The gradient strength of the current pixel is compared with the gradient strength of adjacent pixels along the positive and negative gradient directions, and it is determined whether to retain the current pixel as an edge point according to the comparison result.

5. The method for obtaining bridge vibration with high precision based on computer vision according to claim 4 is characterized in that: When judging whether to retain the current pixel as an edge point according to the comparison result, it includes: If the gradient intensity of the current pixel is the maximum value, retaining the current pixel as an edge point; If the gradient intensity of the current pixel is not the maximum value, the current pixel is suppressed and is not regarded as an edge point.

6. The method for obtaining bridge vibration with high precision based on computer vision according to claim 5 is characterized in that: The method further includes performing edge detection on the grayscale image data using a Canny operator to obtain a plurality of edge points of the bridge to be detected, and calculating the vertical displacement of each edge point one by one: The distance between each edge point and the image frame is calculated one by one, and the distance is recorded as a vertical displacement; wherein the unit of the vertical displacement is a pixel.

7. The method for obtaining bridge vibration with high precision based on computer vision according to claim 6 is characterized in that: When using Zernike moments to calculate sub-pixel accuracy of edge displacement, it includes: Calculating the parameters of the edge point based on the Zernike moment, the parameters include background gray value h, step height k, vertical distance l from the center of the disk to the edge, and angle φ between the vertical line and the x-axis; By calculating three Zernike moments, namely Z'00, Z'11, and Z'20, the background gray value h, step height k, vertical distance l, and angle φ are obtained; The angle φ is obtained by the following formula: The vertical distance l is obtained by the following formula: The step height k is obtained by the following formula:

8. The method for obtaining bridge vibration with high precision based on computer vision according to claim 7 is characterized in that: When using Zernike moments to calculate sub-pixel accuracy of edge displacement, it also includes: Determine the Zernike moment Zn,m before the grayscale image data is rotated and the Zernike moment Z'n,m after the grayscale image data is rotated by an angle φ; wherein the relationship is as shown in the following formula: WITH' n，m =Z n，m exp(-jφ); Let f(x,y) be the rotated image, then: The edge point coordinates (asc, bsc) with sub-pixel accuracy are calculated from the edge point coordinates (a, b):

9. The method for obtaining bridge vibration with high precision based on computer vision according to claim 8 is characterized in that: Calculating a proportional factor corresponding to each of the image data according to the sub-pixel accuracy and the actual size data, and calculating the actual vibration displacement of the bridge to be detected according to the proportional factor, including: The actual size data is the actual physical length D of the bridge to be inspected; The proportional factor is calculated according to the actual physical length D and the pixel length P of the grayscale image data; wherein the proportional factor is obtained by the following formula: Calculate the edge sub-pixel precision displacement according to the edge point coordinates (asc, bsc) with sub-pixel precision; The edge sub-pixel precision displacement is multiplied by a proportional factor to obtain the actual vibration displacement of the bridge to be detected.

10. A system for obtaining bridge vibration with high precision based on computer vision, applied to the method for obtaining bridge vibration with high precision based on computer vision as claimed in any one of claims 1 to 9, characterized in that: include: An extraction module is configured to determine a bridge to be detected, record a vibration video of the bridge to be detected, and extract the vibration video frame by frame to obtain image data of a plurality of bridges to be detected; A processing module is configured to perform grayscale processing on the image data to obtain grayscale image data of the bridge to be detected; An edge detection module is configured to perform edge detection on the grayscale image data using a Canny operator, obtain a plurality of edge points of the bridge to be detected, and calculate the vertical displacement of each edge point one by one; A sub-pixel precision calculation module is configured to calculate the sub-pixel precision of edge displacement using Zernike moments and obtain actual size data of the bridge to be inspected; The proportional factor corresponding to each image data is calculated according to the sub-pixel accuracy and the actual size data, and the actual vibration displacement of the bridge to be detected is calculated according to the proportional factor.

Images