A non-contact measurement method for bridge structure displacement based on improved SIFT algorithm

Abstract

A non-contact measurement method for bridge structure displacement based on an improved SIFT algorithm realizes non-contact and accurate measurement of bridge structure displacement. The specific implementation process is as follows: A. Collect bridge structure video and perform preprocessing; B. Use SIFT algorithm to identify feature points of bridge structure in video; C. Use SuperPoint network model to generate descriptors of bridge structure feature points; D. Use SuperGlue network model to match each frame of video with the bridge structure feature points of the initial image; E. Calculate the displacement of the bridge structure at the corresponding moment based on the matching results. The method disclosed in the present invention improves the SIFT algorithm used for feature point matching, improves the description effect of bridge structure feature points, and improves the accuracy of feature point matching, thereby realizing accurate tracking of bridge structure feature points in video. This method can realize high-precision non-contact bridge structure displacement measurement, and provides a new means for bridge structure deformation monitoring.

Description

Technical Field

[0001] The present invention relates to a non-contact measurement method for bridge structure displacement, which belongs to the field of structural health monitoring and recognizes bridge structure feature points in a tracking video through a SIFT algorithm, improves feature point description and matching methods, and measures bridge structure displacement with high precision. Background Art

[0002] The displacement of a bridge structure directly reflects the stress performance of the structure and can be used to evaluate the safety status of the bridge structure. As an important parameter for analyzing the static and dynamic characteristics of a structure, the displacement of a bridge structure is often used for bearing capacity calculation, modal parameter extraction, influence line identification, etc. Accurately measuring the displacement of a bridge structure is crucial for the health assessment and safe operation and maintenance of the bridge.

[0003] The contact displacement measurement method requires the displacement sensor to contact the target structure, and the displacement measurement is achieved through the change of the contact point position. This method requires a working platform to be set up near the measuring point for installing the sensor. Since it is difficult to set up a working platform under the bridge, the contact method has shortcomings such as difficulty in implementation in bridge displacement measurement. Computer vision, as a non-contact method, has the advantages of simple operation and multi-point measurement when measuring the displacement of bridge structures.

[0004] The SIFT algorithm is a feature point matching algorithm based on computer vision, which can be used to perform non-contact displacement measurement of bridge structures in videos. The algorithm is insensitive to factors such as ambient lighting and video noise, and has good robustness. However, the algorithm uses a fixed distance ratio threshold to judge the matching effect of feature points, which is prone to mismatching and affects the accuracy of bridge structure displacement measurement. Therefore, it is necessary to explore non-contact methods for accurately measuring bridge structure displacement. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention proposes a non-contact measurement method for bridge structure displacement based on an improved SIFT algorithm to improve the measurement accuracy of bridge structure displacement. The specific contents include:

[0006] A non-contact measurement method for bridge structure displacement based on an improved SIFT algorithm comprises the following steps:

[0007] A. Collect and pre-process bridge structure videos;

[0008] B. Use SIFT algorithm to identify the feature points of the bridge structure in the video;

[0009] C. Use SuperPoint network model to generate descriptors of bridge structure feature points;

[0010] D. Use the SuperGlue network model to match each frame of the video with the bridge structure feature points of the initial image;

[0011] E. Calculate the bridge structure displacement at the corresponding time based on the matching results.

[0012] Further, step A specifically includes:

[0013] A1. Arrange targets at the bridge displacement measurement location;

[0014] A2. The monocular camera lens is perpendicular to the side of the bridge to collect video of the bridge structure;

[0015] A3. Crop the video image, keep the area where the displacement measurement points are located, and convert the image into a grayscale image.

[0016] Further, step B specifically includes:

[0017] B1. Perform convolution operation on the image using Gaussian function to obtain images with different blur levels. Blurred images of the same size are grouped together, and the images of adjacent layers in the group are subtracted to construct the Gaussian difference pyramid of the image.

[0018] D(x,y,σ)=L(x,y,kσ)-L(x,y,σ) (1)

[0019] Where L is the scale space function of the blurred image, σ is the scale space factor, and k = 2 1 / s , s represents the number of pictures in each group, s=n+3, and n represents the number of pictures from which feature points need to be extracted.

[0020] B2. Each pixel is compared with all adjacent points in the same layer and the upper and lower adjacent layers. If a point is larger (or smaller) than all adjacent points, then the point is a preliminarily identified SIFT feature point;

[0021] B3. Perform Taylor series expansion on D(X)

[0022]

[0023] Where X = [xy σ] T and X0=[x0 y0 σ0] T Respectively represent the vectors composed of the horizontal and vertical coordinates of the real and preliminarily recognized SIFT feature points and the scale space factors; It represents the correction amount for the initially identified SIFT feature points, which can be obtained by taking the derivative of X equal to 0 in equation (2). D(X) replaces D(X0) in equation (2), and iterates repeatedly until D(X) reaches the predetermined accuracy, thereby correcting the discrete feature points to sub-pixel accuracy;

[0024] B4. Remove SIFT feature points with low contrast and at the edge. The retained feature points need to meet

[0025]

[0026] Where T,γ is the threshold. D xx (x,y),D yy (x,y),D xy (x, y) represents the second-order derivative and second-order mixed partial derivative of the Gaussian difference image pixel in the x-axis and y-axis directions respectively.

[0027] Further, step C specifically includes:

[0028] C1. The shared encoder inputs a H×W grayscale image and transforms the input original image into a (H / 8, W / 8, 128) tensor output through three non-overlapping maximum pooling operations;

[0029] C2. This tensor is input into the descriptor decoder, which outputs the descriptor network through a model similar to the universal matching network. The descriptor is bicubicly interpolated and L2 norm normalized to obtain the descriptors of all pixels in the image;

[0030] C3. Use the pixel coordinates of the SIFT feature point to find the corresponding descriptor in the output result.

[0031] Further, step D specifically includes:

[0032] D1. Input the initial image, the SIFT feature points and SuperPoint descriptors of the current image, and the attention graph neural network encodes the feature points and descriptors into a vector. After multiple iterations of self-attention and cross-attention, the two images obtain their respective matching descriptors;

[0033] D2. The optimal matching module calculates the inner product of the matching descriptors of the two images to obtain the matching score matrix. Add a new row and column to the score matrix, and then use the Sinkhorn algorithm to solve the optimal transmission problem to obtain the distribution matrix Represents the probability that the i-th feature point matches the j-th feature point.

[0034] Further, step E specifically includes:

[0035] E1. Obtain the relative position change of the characteristic points of the bridge structure on the image based on the characteristic point matching results of the initial image and the current image;

[0036] E2. According to the ratio of the length of the real structure to the number of pixels it spans in the image, the displacement of the feature point relative to the initial moment is obtained, and then the displacement time domain and frequency domain curves of the corresponding position of the bridge are obtained.

[0037] The present invention improves the SIFT algorithm used for feature point matching, improves the description effect of bridge structure feature points, and improves the accuracy of feature point matching, thereby achieving accurate tracking of bridge structure feature points in the video. The method can achieve high-precision non-contact bridge structure displacement measurement, providing a new means for bridge structure deformation monitoring.

[0038] The present invention has the following advantages:

[0039] (1) The method disclosed in the present invention is a non-contact measurement method of structural displacement based on computer vision. The structural displacement is calculated by identifying the position change of SIFT feature points on the bridge structure. There is no need to arrange sensors on the surface of the bridge structure and the wires are not arranged wirelessly. The operation is simple and convenient.

[0040] (2) The method disclosed in the present invention can measure micro displacement of structures. The SIFT algorithm can measure micro deformation of bridge structures by performing Taylor series expansion of scale space functions and repeatedly iterating the correction amount of feature point coordinates to obtain sub-pixel accuracy.

[0041] (3) The method disclosed in the present invention improves the SIFT algorithm, introduces the SuperPoint network model to generate feature point descriptors, and improves the feature point description effect; introduces the SuperGlue network model to replace the traditional feature point matching algorithm, and improves the matching accuracy of video image feature points. Compared with the traditional non-contact measurement method, this method achieves a more accurate displacement measurement result. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of the method of the present invention;

[0043] Figure 2a is a schematic diagram of a bridge model of the present invention, Figure 2b This is a schematic diagram of an example of a bridge structure displacement measurement scenario. Figure 2c It is a schematic diagram of another perspective of the bridge structure displacement measurement scenario example;

[0044] Figure 3 It is a schematic diagram of the SuperPoint network model of the present invention;

[0045] Figure 4 is a schematic diagram of the SuperGlue network model of the present invention;

[0046] Figure 5This is a comparison chart of the matching effects between the improved SIFT algorithm of the present invention and the traditional SIFT algorithm;

[0047] Figure 6 It is a time domain result diagram of displacement measured by the present invention;

[0048] Figure 7 It is a displacement frequency domain result diagram measured by the present invention. DETAILED DESCRIPTION

[0049] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0050] The implementation method of the present invention is as follows Figure 1 As shown, a non-contact measurement method for bridge structure displacement based on an improved SIFT algorithm specifically includes the following steps:

[0051] A. Collect bridge structure video and perform preprocessing; specifically including:

[0052] A1. The bridge model and the bridge structure displacement measurement scenario are shown in Figure 2. The displacement measurement test is carried out under the action of the moving load of the model vehicle. Targets are arranged at the required displacement measurement positions of the bridge;

[0053] A2. Use a SONY α6400 monocular camera to shoot a video of the simply supported beam bridge structure. The camera resolution is 1920×1080 and the frame rate is 50fps. The camera is placed about 1m away from the simply supported beam bridge, with the lens perpendicular to the side of the bridge;

[0054] A3. Crop the video image and keep the area where the displacement measurement points are located (image size 496×200, i.e., the area of ​​the original image with horizontal coordinates 700 to 1196 and vertical coordinates 600 to 800) as the image for feature point recognition.

[0055] B. Use SIFT algorithm to identify the characteristic points of the bridge structure in the video; specifically:

[0056] B1. For each frame of video image, use the Gaussian function to perform convolution operation to obtain images with different blur levels. Blurred images of the same size are grouped together, and the images of adjacent layers in the group are subtracted to construct the Gaussian difference pyramid of the image;

[0057] B2. Each pixel is compared with all adjacent points in the same layer and the upper and lower adjacent layers in the Gaussian difference pyramid. If a point is larger (or smaller) than all adjacent points, then the point is a preliminarily identified SIFT feature point;

[0058] B3. Using Taylor series expansion and iteration of correction, the SIFT feature point coordinates are identified to sub-pixel accuracy;

[0059] B4. Remove the SIFT feature points with low contrast and at the edges of the image, and set T = 0.03 and γ = 10.

[0060] C. Use the SuperPoint network model to generate the bridge structure feature point descriptor for each frame of video image; specifically including:

[0061] C1. Input each frame of the video into the SuperPoint model. The network structure of the model is as follows Figure 3 As shown in the figure, through three 2×2 non-overlapping maximum pooling operations, the width and height of the original image are changed to 62 and 25 respectively. After passing through the encoder, the input original image is converted into a tensor output of (25, 62, 128);

[0062] C2. Input the tensor into the descriptor decoder, using a model similar to the universal matching network, and output the descriptor network. Perform bicubic interpolation on the descriptor and perform L2 norm normalization to obtain the descriptors of all pixels in the image;

[0063] C3. Find the corresponding descriptor in the output result by the pixel coordinates of the SIFT feature points identified in each frame of the image;

[0064] D. Use the SuperGlue network model to match each frame of the video with the bridge structure feature points of the initial image; specifically, the following steps are performed:

[0065] D1. Input the initial image, the feature points of the current image and the SuperPoint descriptor into the SuperGlue model. The network structure of the model is as follows: Figure 4 As shown in Figure 2, the feature points and descriptors are encoded into a vector, and the two images are iterated through multiple self-attention and cross-attention to obtain the matching descriptor;

[0066] D2. Calculate the inner product of the matching descriptors of the two images to obtain the matching score matrix. Add a new row and column to the score matrix and use the Sinkhorn algorithm to obtain the distribution matrix;

[0067] E. Obtain the bridge structure displacement at the corresponding time according to the matching results; specifically including:

[0068] E1. Compare the matching results of the traditional SIFT algorithm and the improved SIFT algorithm, and use white lines to connect the matching feature points obtained by the algorithm. The results are as follows: Figure 5 As shown. According to the matching results of the feature points of the initial image and the current image, the relative position change of the feature points of the bridge structure on the image is obtained;

[0069] E2. The ratio of the length of the structure in the real world to the corresponding number of pixels in the video image is calculated to be 1mm:13pixel, and the displacement of the feature point relative to the initial moment is obtained. The displacement time domain comparison results of the corresponding position of the bridge obtained by using this method, the traditional SIFT algorithm and the sensor are as follows: Figure 6 The frequency domain comparison results are shown in Figure 7 shown.

[0070] The contents described in the implementation cases of this specification are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be deemed to be limited to the specific forms described in the implementation cases. The protection scope of the present invention also extends to the equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A non-contact measurement method for bridge structure displacement based on an improved SIFT algorithm, comprising the following steps: A. Collect and pre-process bridge structure videos; B. Use SIFT algorithm to identify the feature points of the bridge structure in the video; C. Use SuperPoint network model to generate descriptors of bridge structure feature points; D. Use the SuperGlue network model to match each frame of the video with the bridge structure feature points of the initial image; E. Calculate the bridge structure displacement at the corresponding time according to the matching results; Step B specifically includes: B1. Perform convolution operation on the image using Gaussian function to obtain images with different blur levels; blurred images of the same size are grouped together, and the images of adjacent layers in the group are subtracted to construct the Gaussian difference pyramid of the image. D(x,y,σ)=L(x,y,kσ)-L(x,y,σ) (1) Where L is the scale space function of the blurred image, σ is the scale space factor, and k = 2 1 / s , s represents the number of pictures in each group, s=n+3, n represents the number of pictures that need to extract feature points; B2. Each pixel is compared with all adjacent points in the same layer and the upper and lower adjacent layers. If a point is larger or smaller than all adjacent points, then the point is a preliminarily identified SIFT feature point; B3. Perform Taylor series expansion on D(X) Where X = [x yσ] T and X0=[x0 y0σ0] T Respectively represent the vectors composed of the horizontal and vertical coordinates of the real and preliminarily recognized SIFT feature points and the scale space factors; It represents the correction amount for the initially identified SIFT feature points, which can be obtained by taking the derivative of formula (2) with respect to X and making it equal to 0. D(X) replaces D(X0) in formula (2), and iterates repeatedly until D(X) reaches the predetermined accuracy, thereby correcting the discrete feature points to sub-pixel accuracy. B4. Remove SIFT feature points with low contrast and at the edge; the retained feature points need to meet the following requirements: In the formula, T,γ is the threshold; D xx (x,y),D yy (x,y),D xy (x, y) represents the second-order derivative and second-order mixed partial derivative of the Gaussian difference image pixel in the x-axis and y-axis directions respectively; Step C specifically includes: C1. The shared encoder inputs a H×W grayscale image and transforms the input original image into a (H / 8, W / 8, 128) tensor output through three non-overlapping maximum pooling operations; C2. This tensor is input into the descriptor decoder, which outputs the descriptor network through a model similar to the universal matching network. The descriptor is bicubicly interpolated and L2 norm normalized to obtain the descriptors of all pixels in the image. C3. Use the pixel coordinates of the SIFT feature point to find the corresponding descriptor in the output result.

2. The non-contact measurement method for bridge structure displacement based on improved SIFT algorithm as claimed in claim 1, characterized in that: Step A specifically includes: A1. Arrange targets at the bridge displacement measurement location; A2. The monocular camera lens is perpendicular to the side of the bridge to collect video of the bridge structure; A3. Crop the video image, keep the area where the displacement measurement points are located, and convert the image into a grayscale image.

3. The non-contact measurement method for bridge structure displacement based on improved SIFT algorithm as claimed in claim 1, characterized in that: Step D specifically includes: D1. Input the initial image, the SIFT feature points and SuperPoint descriptors of the current image, and the attention graph neural network encodes the feature points and descriptors into a vector; after multiple self-attention and cross-attention iterations, the two images obtain their respective matching descriptors; D2. The optimal matching module calculates the inner product of the matching descriptors of the two images to obtain a matching score matrix; adds a new row and column to the score matrix, and then uses the Sinkhorn algorithm to solve the optimal transmission problem to obtain a distribution matrix Represents the probability that the i-th feature point matches the j-th feature point.

4. The non-contact measurement method for bridge structure displacement based on improved SIFT algorithm as claimed in claim 1, characterized in that: Step E specifically includes: E1. Obtain the relative position change of the characteristic points of the bridge structure on the image based on the characteristic point matching results of the initial image and the current image; E2. According to the ratio of the length of the real structure to the number of pixels it spans in the image, the displacement of the feature point relative to the initial moment is obtained, and then the displacement time domain and frequency domain curves of the corresponding position of the bridge are obtained.

Images