Bridge multi-point displacement measurement method and device based on edge detection

Abstract

The application belongs to the technical field of engineering structure monitoring, and discloses a bridge multi-point displacement measurement method and equipment based on edge detection, which comprises the following steps: (1) a general video segmentation large model is used to perform semantic segmentation on the vibration video of the bridge to be measured frame by frame to obtain a bridge structure mask; (2) edge extraction and non-maximum suppression are performed in the structure mask to obtain an edge point set of the bridge structure; (3) the average value of the edge point image coordinates in the neighborhood of the preselected measuring point is calculated based on the edge point set, and then the time sequence of the average edge point coordinates of each measuring point changing with time is obtained, and the bridge measuring point pixel displacement is calculated based on the time sequence; (4) the camera scale factor is calculated based on the object distance between the camera used for obtaining the vibration video and the bridge, and the known camera focal length and pixel size, and then the actual displacement of each measuring point pixel displacement is converted to obtain the bridge multi-point displacement measurement result. The application is not affected by light changes.

Description

Technical Field

[0001] This invention belongs to the technical field of engineering structure monitoring, and more specifically, relates to a method and device for measuring multi-point displacement of bridges based on edge detection. Background Technology

[0002] Bridges experience deflection and multi-point displacement under traffic loads, wind-induced vibrations, earthquakes, and temperature effects. The relevant time-history and frequency-domain characteristics are crucial for structural safety assessments, modal identification, and life prediction. Vision-based bridge displacement measurement methods use bridge video as input, obtaining pixel displacements through image feature tracking or image registration, and then calculating the actual displacement through scaling. In particular, edge-detection-based visual displacement measurement methods have become a research hotspot in recent years due to their advantages, such as not requiring manual targets, the existence of obvious feature points on bridges, and the ability to measure varying displacements.

[0003] However, existing edge detection-based visual displacement measurement methods still face the problem of sensitivity to changes in lighting conditions in engineering settings. Traditional edge detection methods based on gradient thresholds, such as Canny, are highly sensitive to strong non-uniform lighting conditions such as exposure fluctuations, shadows, reflections, highlights, and headlight sweep. Manual thresholds require repeated parameter tuning and exhibit poor robustness. Deep learning methods based on supervised semantic segmentation networks such as U-Net typically require a large amount of labeled data on similar bridges, resulting in long model training and update cycles, poor cross-scene generalization, and difficulty in rapid deployment. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a method and equipment for multi-point displacement measurement of bridges based on edge detection, which aims to solve the problem that the visual displacement measurement of bridges is easily affected by changes in lighting on the engineering site.

[0005] To achieve the above objectives, according to one aspect of the present invention, a method for measuring multi-point displacement of a bridge based on edge detection is provided, comprising the following steps: (1) A general video segmentation model is used to perform semantic segmentation frame by frame on the vibration video of the bridge under test in operation or under external excitation, so as to automatically separate the background from the bridge structure and obtain the bridge structure mask. (2) Perform edge extraction and non-maximum suppression within the structural mask to obtain the set of edge points of the bridge structure; (3) Calculate the average value of the image coordinates of the edge points in the neighborhood of the pre-selected measurement point based on the set of edge points. This involves determining the average edge point coordinates of each measuring point, and then obtaining a time-series sequence of how these coordinates change over time. Based on this time-series sequence, the pixel displacement of the bridge measuring points is calculated. ; (4) Based on the object distance between the camera used to acquire vibration video and the bridge and known camera focal lengths and pixel size Calculate the camera scaling factor, and then shift the pixels at each measurement point. Convert to actual displacement The results of multi-point displacement measurements of the bridge were obtained.

[0006] Furthermore, candidate edge points are located by pixel mutation positions, and candidate edge points are screened according to the edge gradient nonmaximum suppression method, thereby obtaining the set of edge points of the bridge structure.

[0007] Furthermore, the Track Anything model is used to perform semantic segmentation on the vibration video frame by frame.

[0008] Furthermore, in the original image, the gradient of candidate edge points is calculated using the Sobel operator, and the gradients of adjacent pixels are calculated along the normal direction. Based on the calculated gradients, it is determined whether a candidate edge point should be retained as an edge point. After bilinear interpolation sampling of the gradient magnitude along the normal direction for the selected edge points, a one-dimensional quadratic curve fitting is used to calculate the gradient change curve along the normal direction. The image coordinates of the gradient peak position are determined by the obtained curve extrema, i.e., the position of the zero point of the first derivative, and thus the sub-pixel edge coordinates of the edge point are determined. This enables edge positioning from pixel level to sub-pixel level.

[0009] Furthermore, the Sobel operators for the horizontal and vertical directions are as follows:

[0010]

[0011] The Sobel operator is applied to calculate the horizontal and vertical gradients of the image, and then the gradient magnitudes are calculated. and direction :

[0012] .

[0013] Furthermore, based on the pre-selected measurement point locations within the set of edge points, and assuming that the bridge displacement is perpendicular to the edge direction, the average value of the edge point image coordinates within the neighborhood of each measurement point is calculated. The time series sequence of the average edge point coordinates of each measuring point is obtained as a function of time, and then the pixel displacement of the bridge measuring points is obtained by the pixel coordinate changes. .

[0014] Furthermore, by measuring the object distance between the camera and the bridge... and known camera focal lengths and pixel size Calculate the camera scaling factor and shift the pixels at each measurement point accordingly. Convert to actual displacement The results of multi-point displacement measurement of the bridge were obtained. .

[0015] The present invention also provides a bridge multi-point displacement measurement system based on edge detection. The system includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it performs the bridge multi-point displacement measurement method based on edge detection as described above.

[0016] The present invention also provides a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the bridge multi-point displacement measurement method based on edge detection as described above.

[0017] In summary, compared with the prior art, the bridge multi-point displacement measurement method and equipment based on edge detection provided by the present invention have the following advantages: 1. A general video segmentation model is used to perform semantic segmentation frame by frame on the vibration video of the bridge under test under operation or external excitation to automatically separate the background from the bridge structure and obtain a bridge structure mask. In this segmentation process, the threshold is manually set and the labeled dataset is used for training. It can maintain high segmentation stability even under strong lighting changes and interference from shadows and reflections. It is not affected by lighting changes and can achieve high-precision synchronous measurement of multiple measurement points. It can achieve robust, sub-pixel-level non-contact measurement of multi-point displacement of bridge structure without the need for manual targets or scene-specific training data.

[0018] 2. Because the general video segmentation model is trained on a large amount of standard data, including scenes with various lighting conditions, shooting angles, and camera resolutions, it can obtain stable and reliable bridge segmentation results under varying lighting conditions, even for unprecedented bridge vibration measurement scenarios, without requiring image thresholding, additional labeled data, or training.

[0019] 3. The method proposed in this invention is simple to calculate and easy to apply, and can accurately calculate the multi-point displacement of the bridge under varying illumination conditions. Attached Figure Description

[0020] Figure 1 This is a flowchart of a bridge multi-point displacement measurement method based on edge detection provided in Embodiment 1 of the present invention; Figure 2This is a schematic diagram of a vibration video frame; Figure 3 This is a schematic diagram of the segmentation results of the general visual segmentation large model for vibration video frames; Figure 4 This is a schematic diagram of a bridge structure mask image obtained from a general visual segmentation model of vibration video frames. Figure 5 This is a schematic diagram of the simply supported beam structure and the positions of the three measuring points in vibration measurement according to Embodiment 2 of the present invention; Figure 6 (a), (b), and (c) are schematic diagrams of video frames of a simply supported beam structure under different lighting conditions during vibration measurement. Figure 7 (a) and (b) in the figure are comparison results of the displacement of point 1 obtained by the Canny method and the general visual segmentation large model and the measurement results of the laser displacement sensor in the multi-point displacement measurement experiment of simply supported beam. Figure 8 (a) and (b) in the figure are comparison figures of the point 2 displacement obtained by the Canny method and the general visual segmentation large model and the measurement results of the laser displacement sensor in the multi-point displacement measurement experiment of simply supported beam; Figure 9 (a) and (b) in the figure are schematic diagrams comparing the point 3 displacement obtained by the Canny method and the general visual segmentation large model with the measurement results of the laser displacement sensor in the multi-point displacement measurement experiment of a simply supported beam. Figure 10 This is a schematic diagram of a bridge multi-point displacement measurement device based on edge detection under varying illumination conditions, provided in Embodiment 3 of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0022] This invention provides a method for measuring the multi-point displacement of a bridge based on edge detection. In the process of measuring the multi-point displacement of a bridge, the method first separates the main structure from the background area in each frame of the vibration video using a general semantic segmentation model. Then, candidate edge points are located by pixel mutation positions. Candidate edge points are screened according to the edge gradient non-maximum suppression method and the sub-pixel coordinates are calculated. Finally, the average value of the image coordinates of the edge points in the neighborhood of each measurement point is calculated and the actual displacement of the bridge at multiple points is obtained by using a scaling factor.

[0023] The measurement method mainly includes the following steps: Step 1: Use a general video segmentation model to perform semantic segmentation frame by frame on the vibration video of the bridge under test in operation or under external excitation, so as to automatically separate the background from the bridge structure and obtain the bridge structure mask.

[0024] The segmentation process does not rely on manually set thresholds or labeled datasets for training, and it maintains high segmentation stability even under conditions of strong lighting changes and interference from shadows and reflections. One implementation method can use models such as Track Anything.

[0025] Before step one, acquire vibration videos of the bridge under test during operation or under external excitation, and fix the camera in place for non-contact imaging acquisition.

[0026] Step 2: Perform edge extraction and non-maximum suppression within the structural mask to obtain the set of edge points of the bridge structure.

[0027] Since the mask image has pixels of 1 in the main part of the bridge structure and pixels of 0 in complex background areas, all candidate edge points can be obtained by finding the locations of pixel abrupt changes. Subsequently, in the original image, the gradient of candidate edge points is calculated using the Sobel operator, and the gradients of adjacent pixels are calculated along the normal direction. If the gradient of a candidate edge point is the largest, it is retained as an edge point; otherwise, it is suppressed and considered a non-edge point. Finally, after bilinear interpolation sampling of the gradient magnitude along the normal direction for the filtered edge points, a one-dimensional quadratic curve fitting is used to calculate the gradient change curve along the normal direction. The image coordinates of the gradient peak position are determined by the obtained curve extrema, i.e., the position of the zero point of the first derivative, and thus the sub-pixel edge coordinates of the edge point are determined. This enables pixel-level to sub-pixel-level edge positioning refinement.

[0028] Step 3: Calculate the average value of the image coordinates of the edge points within the neighborhood of the pre-selected measurement point based on the edge point set. This involves determining the average edge point coordinates of each measuring point, and then obtaining a time-series sequence of how these coordinates change over time. Based on this time-series sequence, the pixel displacement of the bridge measuring points is calculated. .

[0029] Specifically, based on the pre-selected measurement point locations within the set of edge points, and assuming that the bridge displacement is mainly perpendicular to the edge direction and has a small amplitude, the average value of the edge point image coordinates in the neighborhood of each measurement point is calculated. That is, the average edge point coordinates of the measuring points, and then the time series sequence of the average edge point coordinates of each measuring point changing with time is obtained. Based on the time series sequence, the pixel displacement of the bridge measuring points is obtained by the pixel coordinate changes. .

[0030] Step 4: Based on the object distance between the camera used to acquire the vibration video and the bridge. and known camera focal lengths and pixel size Calculate the camera scaling factor, and then shift the pixels at each measurement point. Convert to actual displacement The results of multi-point displacement measurements of the bridge were obtained.

[0031] The present invention also provides a bridge multi-point displacement measurement system based on edge detection. The system includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it performs the bridge multi-point displacement measurement method based on edge detection as described above.

[0032] The present invention also provides a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the bridge multi-point displacement measurement method based on edge detection as described above.

[0033] The present invention will be further described in detail below with reference to specific embodiments.

[0034] Example 1 Please see Figure 1 Embodiment 1 of the present invention provides a method for measuring the multi-point displacement of a bridge based on edge detection under varying illumination conditions. The process is as follows: Step 101: Acquire vibration videos of the bridge under test during operation or under external excitation. The camera is fixedly deployed for non-contact imaging acquisition.

[0035] In practical engineering applications, a camera is fixed in an unobstructed position under the bridge to continuously record video, and the acquired image data is compiled into an image set. The changes in the bridge structure's edges in each frame can provide displacement information at that location. Image processing techniques are used to analyze the image sequence frame by frame, providing data support for subsequent displacement calculations.

[0036] Step 102: Use a general video segmentation model to perform semantic segmentation on the vibration video frame by frame to automatically separate the complex background from the bridge structure and obtain an accurate bridge structure mask image.

[0037] In this embodiment, to overcome the structural edge extraction errors caused by environmental factors such as changes in illumination, a general video segmentation model is employed. First, the bridge location in the first frame of the vibration video is selected to provide the necessary prompts for the general video segmentation model, resulting in a bridge segmentation result in the first frame. Then, based on the image features in the first frame, the general video segmentation model segments the bridge in all subsequent video frames, obtaining accurate bridge structure mask images for all video frames. In practice, models such as Track Anything can be used to implement the general video segmentation model. Since the general video segmentation model is trained on a large amount of standard data, including various illumination, shooting angles, and camera resolution scenarios, even in unprecedented bridge vibration measurement scenarios, the general video segmentation model can obtain stable and reliable bridge segmentation results under varying illumination conditions without requiring image thresholds, additional labeled data, or training. The vibration video frames, their corresponding segmentation results, and the bridge structure mask images are shown below. Figure 2 , Figure 3 and Figure 4 As shown.

[0038] Step 103: Perform edge extraction and non-maximum suppression within the structure mask to obtain a high-precision set of edge points for the bridge structure.

[0039] In this design, the mask image has pixels of 1 (white) in the main body of the bridge structure and pixels of 0 (black) in complex background areas. Therefore, all candidate edge points can be obtained by finding the locations of pixel abrupt changes. Subsequently, in the original image, the gradient of candidate edge points is calculated using the Sobel operator, and the gradients of adjacent pixels are calculated along the normal direction. If the gradient of a candidate edge point is the largest, it is retained as an edge point; otherwise, it is suppressed and considered a non-edge point. The Sobel operators for the horizontal and vertical directions are as follows:

[0040]

[0041] The Sobel operator is applied to calculate the horizontal and vertical gradients of the image, and then the gradient magnitudes are calculated. and direction :

[0042]

[0043] Finally, the filtered edge points Along the normal direction Bilinear interpolation sampling of gradient magnitude is performed to obtain three neighboring points. Gradient magnitudes of [1, 0, +1]: The gradient change curve of the normal direction is calculated by using quadratic curve fitting. And through the extreme values ​​of the curve, that is, the positions of the zeros of the first derivative. Determine the image coordinates of the gradient peak location. This allows for the determination of the sub-pixel edge coordinates of the edge point. This enables pixel-level to sub-pixel-level edge positioning refinement.

[0044] Step 104: Calculate the pixel displacement of multiple points on the bridge based on the edge detection results.

[0045] Based on the pre-selected measurement point locations within the edge point set, and assuming that the bridge displacement is mainly perpendicular to the edge direction and the inter-frame displacement is small, the average value of the edge point image coordinates in the neighborhood of each measurement point is calculated. The time series sequence of the average edge point coordinates of each measuring point is obtained as a function of time, and then the pixel displacement of the bridge measuring points is obtained by the pixel coordinate changes. .

[0046] Step 105: Complete the pixel-to-actual-scale conversion to obtain the actual displacement of multiple points on the bridge.

[0047] The distance between the camera and the bridge (Obtainable via laser rangefinder) and the known camera focal length. and pixel size Calculate the camera scaling factor and shift the pixels at each measurement point accordingly. Convert to actual displacement The results of multi-point displacement measurement of the bridge were obtained. .

[0048] Example 2 Embodiment 2 of the present invention provides a method for measuring the multi-point displacement of a bridge based on edge detection under varying illumination conditions. Based on Embodiment 1, the method is demonstrated in a real-world scenario for measuring the multi-point displacement of a bridge.

[0049] The following example, using a simply supported beam model for multi-point displacement measurement, illustrates the application of the method provided by this invention in multi-point displacement measurement of bridges.

[0050] A 2m long simply supported beam model, fixed at one end and simply supported at the other, was subjected to sinusoidal external excitation via an exciter connected to the left third of the beam. Vibration videos of the simply supported beam structure were filmed using an industrial camera at a distance of 2.1m. Three measuring points were set on the simply supported beam: the mid-span point and two points on either side 30cm from the mid-span. A schematic diagram of the simply supported beam structure and the measuring point locations is shown below. Figure 5As shown, the video frame rate is 50fps, and the image resolution is 2000. 250 pixels.

[0051] The lighting conditions were altered during the experiment by changing the status of the indoor lights' switches and the position of the lighting equipment. Vibration video frames under different lighting conditions are shown below. Figure 6 As shown in the figure. Subsequently, edge detection was performed on the vibration video using the Canny edge detection method and a general video segmentation large model, and the displacement extraction results were calculated. Finally, the results were compared with the data from the laser displacement sensor, and the standard root mean square error (NRMSE) was calculated using the following formula: ; Where, x i For visual sensing data, y i For laser displacement sensor data, y max y represents the maximum value of the laser displacement sensor data. min denoted as the minimum value of the laser displacement sensor data, and n as the data volume.

[0052] The displacement measurement results of points 1, 2, and 3 are as follows: Figure 7 , Figure 8 and Figure 9 As shown in Table 1, the error calculation results show that under varying illumination conditions, the dynamic deflection measurement results of the Canny edge detection method deviate from the actual value, while the edge detection method based on the general video segmentation large model proposed in this embodiment can accurately extract the edge of the simply supported beam, and the measurement results are consistent with those of the laser displacement sensor.

[0053] Table 1. NRMSE values ​​of point 1, point 2, and point 3 displacements obtained by the Canny method and the general visual segmentation model, compared with the laser displacement sensor measurement results.

[0054] This embodiment provides a method for measuring multi-point bridge displacement based on edge detection under varying illumination conditions. During bridge edge detection, this method is unaffected by changes in ambient illumination, allowing for accurate edge calculation and thus precise multi-point bridge displacement. The proposed method is computationally simple, easy to apply, and can accurately calculate multi-point bridge displacement under varying illumination conditions.

[0055] Example 3 like Figure 10 The diagram shown is a schematic of a bridge multi-point displacement measurement device based on edge detection under varying illumination conditions, according to Embodiment 3 of the present invention. This device includes one or more processors 21 and a memory 22. Figure 10 Take a processor 21 as an example.

[0056] Processor 21 and memory 22 can be connected via a bus or other means. Figure 10 Taking the example of a connection between China and Israel via a bus.

[0057] The memory 22, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the bridge multi-point displacement measurement method based on edge detection under varying illumination conditions in the above embodiment. The processor 21 executes the bridge multi-point displacement measurement method based on edge detection under varying illumination conditions by running the non-volatile software program and instructions stored in the memory 22.

[0058] Memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 22 may optionally include memory remotely located relative to processor 21, which can be connected to processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0059] The program instructions / modules are stored in the memory 22. When executed by one or more processors 21, they perform the bridge multi-point displacement measurement method based on edge detection under illumination changes as described in the above embodiments. For example, they perform the above-described... Figures 1 to 7 The steps shown.

[0060] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for measuring multi-point displacement of a bridge based on edge detection, characterized in that, The steps are as follows: (1) A general video segmentation model is used to perform semantic segmentation frame by frame on the vibration video of the bridge under test in operation or under external excitation, so as to automatically separate the background from the bridge structure and obtain the bridge structure mask. (2) Perform edge extraction and non-maximum suppression within the structural mask to obtain the set of edge points of the bridge structure; (3) Calculate the average value of the image coordinates of the edge points in the neighborhood of the pre-selected measurement point based on the set of edge points. This involves determining the average edge point coordinates of each measuring point, and then obtaining a time-series sequence of how these coordinates change over time. Based on this time-series sequence, the pixel displacement of the bridge measuring points is calculated. ; (4) Based on the object distance between the camera used to acquire vibration video and the bridge and known camera focal lengths and pixel size Calculate the camera scaling factor, and then shift the pixels at each measurement point. Convert to actual displacement The results of multi-point displacement measurements of the bridge were obtained.

2. The bridge multi-point displacement measurement method based on edge detection as described in claim 1, characterized in that: Candidate edge points are located by pixel mutation positions, and candidate edge points are screened using the edge gradient nonmaximum suppression method, thereby obtaining the set of edge points of the bridge structure.

3. The bridge multi-point displacement measurement method based on edge detection as described in claim 1, characterized in that: The Track Anything model was used to perform semantic segmentation on each frame of the vibration video.

4. The bridge multi-point displacement measurement method based on edge detection as described in claim 2, characterized in that: In the original image, the gradient of candidate edge points is calculated using the Sobel operator, and the gradients of adjacent pixels are also calculated along the normal direction. The calculated gradients determine whether a candidate edge point should be retained as an edge point. For the filtered edge points, bilinear interpolation sampling of the gradient magnitude is performed along the normal direction. A one-dimensional quadratic curve fitting is then used to calculate the gradient change curve along the normal direction. The extreme values ​​of the obtained curves, i.e., the zero points of the first derivative, are used to determine the image coordinates of the gradient peak position, thereby determining the sub-pixel edge coordinates of the edge point. This enables edge positioning from pixel level to sub-pixel level.

5. The bridge multi-point displacement measurement method based on edge detection as described in claim 4, characterized in that: The Sobel operators for the horizontal and vertical directions are as follows: The Sobel operator is applied to calculate the horizontal and vertical gradients of the image, and then the gradient magnitudes are calculated. and direction : 。 6. The bridge multi-point displacement measurement method based on edge detection as described in claim 5, characterized in that: Based on the pre-selected measurement point locations within the set of edge points, and assuming that the bridge displacement is perpendicular to the edge direction, the average value of the edge point image coordinates within the neighborhood of each measurement point is calculated. The time series sequence of the average edge point coordinates of each measuring point is obtained as a function of time, and then the pixel displacement of the bridge measuring points is obtained by the pixel coordinate changes. .

7. The bridge multi-point displacement measurement method based on edge detection as described in claim 6, characterized in that: The distance between the camera and the bridge and known camera focal lengths and pixel size Calculate the camera scaling factor and shift the pixels at each measurement point accordingly. Convert to actual displacement The results of multi-point displacement measurement of the bridge were obtained. .

8. A bridge multi-point displacement measurement system based on edge detection, characterized in that: The system includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it performs the bridge multi-point displacement measurement method based on edge detection as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the bridge multi-point displacement measurement method based on edge detection as described in any one of claims 1-7.

Images