Structure large displacement estimation method based on Canny-Hough transformation and KLT optical flow

Abstract

The invention discloses a structure large displacement estimation method based on Canny-Hough transformation and KLT optical flow, and relates to the related field of signal processing and vision measurement, and the method comprises the steps: collecting and preprocessing structure vibration data; carrying out integer pixel displacement estimation on the basis of Canny-Hough transformation; reconstructing the video; performing sub-pixel displacement estimation based on the KLT optical flow; and physical displacement. According to the method, the limitation of a single identification method in identifying different pixel levels (pixel level and sub-pixel level) is overcome, the measurement precision of the large displacement of the structure is remarkably improved, and an estimation error caused when an image pyramid is used for large displacement tracking is avoided. The research provides a new target-free large-displacement motion estimation scheme for structural vibration measurement based on computer vision, and has important reference value.

Description

Technical Field

[0001] The present invention relates to the technical fields of bridge structures and visual measurement, and particularly relates to a large displacement estimation of structural vibration. Background Art

[0002] There are many physical indicators for evaluating the static and dynamic characteristics of structures, such as bearing capacity, deflection, deformation, modal parameters, etc. However, these indicators can basically be converted through the displacement of the structure. Therefore, displacement can be used as an important indicator for the state assessment and performance evaluation of structures. The displacement measurement of structural vibration is also one of the keys to structural health monitoring. Traditional structural displacement measurement is usually completed by installing contact sensors at specific positions on the structure. This method has disadvantages such as high cost, difficult installation, limited measuring points, low accuracy, and poor real-time performance. With the development of computer vision technology, vision-based non-contact structural measurement methods have been widely applied and concerned in the engineering field. As an alternative, the structural displacement measurement technology based on computer vision is relatively low-cost, flexible, and can provide the ability of multi-point simultaneous measurement.

[0003] In recent years, non-contact structural vibration measurement methods have mainly focused on template matching, digital image correlation, feature point matching, optical flow, and edge detection. Among them, template matching and digital image correlation are mainly applicable to target recognition with artificial markings, and have problems of low computational efficiency and large memory consumption. For the feature point matching method, the displacement is mainly identified by the distance between descriptors, and its recognition accuracy is easily affected by illumination conditions and the like. The optical flow method and the edge detection algorithm can realize target-free vibration measurement according to the characteristics of the structure itself, so they have attracted much attention. Currently, the most widely used optical flow algorithm is the Kanade-Lucas-Tomasi (KLT) optical flow method. However, the KLT optical flow method is based on the small motion assumption and is applicable to small displacement motion estimation and not applicable to large displacements. Although the measurement range can be expanded through an image pyramid to cope with large displacement motion, image distortion may occur during the pyramid downsampling process, affecting the accuracy of optical flow estimation. Since edges are usually closely related to the shape and contour of an object, edge detection is particularly suitable for pixel-level displacement recognition, that is, large displacement motion recognition. However, the edge detection algorithm shows low robustness when identifying small displacement motion, especially sub-pixel level motion estimation. In addition, large displacement refers to a large pixel displacement of an object's motion between consecutive frames, which can reach the pixel level. In visual measurement, the situation where an object has a pixel-level or above motion is mainly attributed to the influence of the sampling frequency and resolution of the acquisition device. If a low-sampling device is used or the image of the structure is taken with high resolution, then the structural motion captured between consecutive frames will show a large pixel displacement. Secondly, small displacement motion, that is, sub-pixel level displacement, will definitely occur during the structural vibration process.

[0004] In summary, for the case where the motion of the structure includes both pixel-level and sub-pixel-level displacements, especially in applications in large-displacement or fast-motion scenarios, a single recognition method often has limitations. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a large-displacement estimation method for structures based on Canny-Hough transform and KLT optical flow, so as to improve the large-displacement measurement accuracy of structures. The specific scheme is as follows:

[0006] In the first aspect, the present application discloses a large-displacement estimation method for structures based on Canny-Hough transform and KLT optical flow, including: Structural vibration data acquisition and preprocessing; Whole-pixel displacement estimation based on Canny-Hough transform; Video reconstruction; Sub-pixel displacement estimation based on KLT optical flow; Physical displacement.

[0007] Optionally, the structural vibration data acquisition and preprocessing include: Calibrate the acquisition device to obtain the internal and external parameters of the camera and the scale factor of the physical displacement to be converted; Collect the original video of the structural vibration and correct it to obtain the corrected vibration video image; Crop and rotate the video to obtain the vibration video of the target area to be recognized.

[0008] Optionally, the whole-pixel displacement estimation based on Canny-Hough transform includes: Segment the vibration video to obtain a sequence of frame images; Use the Canny operator to perform edge detection on the image to obtain the edge features of all frames; Use the Hough transform to extract the straight lines and their pixel coordinates in the edge features to achieve whole-pixel-level displacement estimation.

[0009] Optionally, the use of the Canny operator to perform edge detection on the image to obtain the edge features of all frames includes: Use the Gaussian function to construct a filter kernel and perform convolution operation on the frame image to achieve smoothing; The expression of the Gaussian function is as follows: In the formula, a and b represent the horizontal and vertical coordinates of the image pixels; σ represents the standard deviation of the Gaussian function, which controls the filtering degree. Calculate the gradient of the smoothed image to obtain the gradient intensity and gradient amplitude; The expression for gradient calculation is as follows: In the formula, G x and G y respectively represent templates for detecting gradient changes in the x and y directions; G and θ respectively represent gradient intensity and direction. Finally, perform non-maximum suppression and threshold screening operations on the detected gradient intensity map to filter out noise and pixels with small gradients, and obtain clearer and continuous edge features.

[0010] Optionally, extracting straight lines and their pixel coordinates in the edge features using the Hough transform to achieve displacement estimation at the whole-pixel level includes: Extracting straight lines in the edge features through the Hough transform; The expression of the Hough transform is as follows: ρ = xcosθ + ysinθ In the formula, (ρ, θ) represents the coordinates in the parameter space. After the Hough transform, each edge feature point (x, y) in the image space is mapped to the point (ρ, θ) in the parameter space, and then the parameters corresponding to the threshold are found through statistical characteristics, and these parameters represent the straight lines in the edge feature map. Obtaining the pixel coordinates of the straight lines in the edge feature map through the extracted straight line parameters; Estimating the whole-pixel displacement through the pixel coordinates.

[0011] Optionally, the video reconstruction includes: Calculating the whole-pixel displacement between consecutive frames to obtain the time course of the whole-pixel displacement; Performing translational transformation processing on the images of the target region to be recognized according to this time course of the whole-pixel displacement to reconstruct the video.

[0012] Optionally, the sub-pixel displacement estimation based on KLT optical flow includes: Using a feature detector to detect feature points in the reference frame; Tracking the motion of this feature point in the current frame using the Kanade-Lucas-Tomasi (KLT) optical flow to estimate the optical flow between consecutive frames; Extracting the sub-pixel displacement between consecutive frames through the optical flow to achieve sub-pixel displacement estimation.

[0013] Optionally, the physical displacement includes: Adding the whole-pixel and sub-pixel displacements to obtain the accurate pixel displacement; Converting the accurate pixel displacement into physical displacement through the scale factor to obtain the large displacement of the accurate structural vibration.

[0014] The beneficial effects of this application are as follows: collecting and preprocessing structural vibration data; estimating the integer-pixel displacement based on the Canny-Hough transform; video reconstruction; estimating the sub-pixel displacement based on the KLT optical flow; physical displacement. It can be seen that this application realizes a method combining the Canny-Hough transform and the KLT optical flow to identify structural vibrations that contain both pixel-level and sub-pixel-level displacements, thereby improving the measurement accuracy of large structural displacements and enhancing the recognition robustness. In addition, this application realizes the recognition of large-pixel displacements of the structure by restricting pixel-level motion to the sub-pixel level through translational transformation processing of the image. This processing method avoids image distortion and can effectively guarantee the recognition performance of large-displacement motion; this application overcomes the limitations of single recognition methods and significantly improves the measurement accuracy of large structural displacements. Brief Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0016] Figure 1 It is a flowchart of a method for estimating large structural displacements based on the Canny-Hough transform and the KLT optical flow disclosed in this application; Figure 2 It is a schematic diagram of a scene for estimating large structural displacements based on the Canny-Hough transform and the KLT optical flow disclosed in this application; Figure 3 It is a schematic diagram of edge detection of an image using the Canny operator disclosed in this application; Figure 4 It is a schematic diagram of detecting straight lines in edge features using the Hough transform disclosed in this application; Figure 5 It is a schematic diagram of estimating the integer-pixel displacement based on the Canny-Hough transform disclosed in this application; Figure 6 It is a schematic diagram of translational transformation of an image of a target area to be recognized disclosed in this application; Figure 7 It is a flowchart of accurate physical displacement estimation based on the Canny-Hough transform and the KLT optical flow disclosed in this application; Figure 8 It is a comparison diagram of the results of estimating large structural displacements based on the Canny-Hough transform and the KLT optical flow disclosed in this application; Detailed Embodiments

[0017] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention:

[0018] There are many physical indicators for evaluating the static and dynamic characteristics of a structure, such as bearing capacity, deflection, deformation, modal parameters, etc. However, these indicators can basically be converted through the displacement of the structure. Therefore, displacement can be used as an important indicator for the state evaluation and performance evaluation of the structure. The displacement measurement of structural vibration is also one of the keys to structural health monitoring. Traditional structural displacement measurement is usually completed by installing contact sensors at specific positions on the structure. This method has the disadvantages of high cost, difficult installation, limited measurement points, low accuracy, and poor real-time performance. With the development of computer vision technology, vision-based non-contact structural measurement methods have been widely used and concerned in the engineering field. As an alternative, the computer vision-based structural displacement measurement technology is relatively low-cost, flexible, and can provide the ability to measure multiple points simultaneously.

[0019] In recent years, non-contact structural vibration measurement methods have mainly focused on template matching, digital image correlation, feature point matching, optical flow, and edge detection. Among them, template matching and digital image correlation are mainly applicable to target recognition with artificial markings and have problems of low computational efficiency and large memory consumption. For feature point matching methods, displacement is mainly recognized through the distance between descriptors, and its recognition accuracy is easily affected by lighting conditions and the like. The optical flow method and edge detection algorithms can achieve target-free vibration measurement based on the characteristics of the structure itself, so they have attracted much attention. The most widely used optical flow algorithm at present is the Kanade-Lucas-Tomasi (KLT) optical flow method. However, the KLT optical flow method is based on the small motion assumption and is applicable to small displacement motion estimation and not applicable to large displacements. Although the measurement range can be expanded through image pyramids to cope with large displacement motion, image distortion may occur during the pyramid downsampling process, affecting the accuracy of optical flow estimation. Since edges are usually closely related to the shape and contour of an object, edge detection is particularly suitable for pixel-level displacement recognition, that is, large displacement motion recognition. However, edge detection algorithms show low robustness when recognizing small displacement motion, especially sub-pixel-level motion estimation. In addition, large displacement refers to a relatively large pixel displacement of an object's motion between consecutive frames, capable of reaching the pixel level. In visual measurement, the situation where an object has a motion at the pixel level and above is mainly attributed to the influence of the sampling frequency and resolution of the acquisition device. If a low-sampling device is used or the image of the captured structure has high resolution, then the structural motion captured between consecutive frames will be manifested as a large pixel displacement. Secondly, during the structural vibration process, there will definitely be a situation of small displacement motion, that is, sub-pixel-level displacement. To sum up, when the motion of the structure includes both pixel-level and sub-pixel-level displacements, especially in applications in large displacement or fast motion scenarios, single recognition methods often have limitations.

[0020] Therefore, the present application correspondingly provides a method for estimating large displacements of a structure based on Canny-Hough transform and KLT optical flow, so as to improve the measurement accuracy of large displacements of the structure and improve the robustness of real-time cable force recognition. It overcomes the limitations of single recognition methods in recognizing different pixel levels (pixel level, sub-pixel level), etc.

[0021] See Figure 1 As shown, the embodiment of the present application discloses a method for estimating large displacements of a structure based on Canny-Hough transform and KLT optical flow, including: Step S11: Collect and preprocess structural vibration data.

[0022] In this embodiment, the acquisition of structural vibration data collection and preprocessing includes: calibrating the acquisition device to obtain the internal and external parameters of the camera and the scale factor K of the physical displacement to be converted; collecting the original video of the structural vibration and correcting it to obtain the corrected vibration video image; cropping and rotating the video to obtain the vibration video of the target area to be recognized. For example Figure 2 The schematic diagram of a structural large displacement estimation scenario based on Canny-Hough transform and KLT optical flow is shown. The video acquisition device is a Canon EOS-1D X Mark II camera with a resolution of 1920×1080 pixels and a frame rate of 30 fps. To ensure the reliability of the computer vision system, the distortion of the camera lens must be considered. Therefore, it is necessary to correct the collected structural vibration video before preprocessing the video image. First, the Zhang-Zhengyou chessboard calibration method is used to calibrate the camera to obtain the internal and external parameters of the camera, and the scale factor method is used to calculate the ratio between the physical displacement and the pixel displacement (unit: mm / pixel) so that the subsequent pixel displacement can be converted into the physical displacement. Secondly, the original video of the frame structure vibration is collected by this camera, and the total acquisition duration is 9 seconds. The structural vibration is mainly caused by artificial excitation. Then, the collected original vibration video is corrected using the calibrated parameters to obtain the corrected vibration video image. Finally, the video is cropped and rotated to obtain the vibration video of the target area to be recognized (node 1, node 2).

[0023] Step S12: Whole-pixel displacement estimation based on Canny-Hough transform.

[0024] In this embodiment, the whole-pixel displacement estimation based on Canny-Hough transform includes: segmenting the vibration video to obtain a sequence of frame images; using the Canny operator to perform edge detection on the images to obtain the edge features of all frames; using the Hough transform to extract the straight lines and their pixel coordinates in the edge features to achieve whole-pixel level displacement estimation. For example Figure 3 The schematic diagram of using the Canny operator to perform edge detection on an image disclosed in this application is shown. The specific steps are as follows (1) Construct a filter kernel using the Gaussian function and perform a convolution operation on the frame image to achieve smoothing. The expression of the Gaussian function is as follows In the formula, a and b represent the horizontal and vertical coordinates of the image pixels; σ represents the standard deviation of the Gaussian function, which controls the filtering degree. (2) Calculate the gradient of the smoothed image to obtain the gradient intensity and gradient amplitude. The expression of the gradient calculation is as follows In the formula, Gx and G y respectively represent templates for detecting gradient changes in the x and y directions; G and θ represent gradient intensity and direction respectively. (3) Finally, perform non-maximum suppression and threshold screening operations on the detected gradient intensity map to filter out noise and pixels with small gradients, and obtain clearer and continuous edge features.

[0025] For example Figure 4 is a schematic diagram of detecting a straight line in edge features using the Hough transform disclosed in this application. In the figure, the magenta straight line represents the straight line with the most statistical significance detected by the Hough transform. Its corresponding feature points are not only strictly located on the same straight line, but also the number of feature points on this straight line is the largest among all detected straight lines. In this way, the straight line parameters with the most features can be extracted to realize the estimation of the integer-pixel displacement of the structure. The specific steps are as follows: (1) Extract the straight line in the edge features through the Hough transform. The expression of the Hough transform is as follows: ρ = xcosθ + ysinθ In the formula, (ρ, θ) represents the coordinates in the parameter space. After the Hough transform, each edge feature point (x, y) in the image space is mapped to the point (ρ, θ) in the parameter space, and then the parameters corresponding to the threshold are found through statistical characteristics. These parameters represent the straight lines in the edge feature map. (2) Obtain the pixel coordinates of the straight line in the edge feature map through the extracted straight line parameters; (3) Estimate the integer-pixel displacement through the pixel coordinates. For example Figure 5 is a schematic diagram of integer-pixel displacement estimation based on the Canny-Hough transform disclosed in this application.

[0026] Step S13: Video reconstruction.

[0027] In this embodiment, the video reconstruction includes: calculating the integer-pixel displacement between consecutive frames to obtain the time course of the integer-pixel displacement; performing a translation transformation on the image of the target region to be recognized according to this time course of the integer-pixel displacement to reconstruct the video. For example Figure 6 is a schematic diagram of image translation transformation of the target region to be recognized disclosed in this application. In the figure, it is assumed that the red solid box represents the position R of the target structure in the nth frame n (x 0 , y 0 ), and the green solid box represents the true position R of the target structure tracked in the (n + 1)th frame n+1 (x 0 + △x, y 0+△y). The blue dashed box represents the integer pixel displacement coordinates extracted based on the Canny-Hough transform, denoted as Then, perform a translation transformation on the ROI image in the (n + 1)-th frame to obtain the position of the new target structure in the (n + 1)-th frame That is, the image of the (n + 1)-th frame is translated by pixel units, restricting the structural movement within one pixel. Finally, use the KLT optical flow to estimate the precise sub-pixel displacement (δx, δy) of the (n + 1)-th frame after the translation transformation. Therefore, the precise pixel displacement (Dx, Dy) between consecutive frames is: where the values of |δx| and |δy| are both less than one pixel.

[0028] Step S14: Estimate the sub-pixel displacement based on the KLT optical flow.

[0029] In this embodiment, the sub-pixel displacement estimation based on the KLT optical flow includes: using a feature detector to detect feature points in the reference frame; adopting the Kanade-Lucas-Tomasi (KLT) optical flow to track the movement of these feature points in the current frame, thereby estimating the optical flow between consecutive frames; and extracting the sub-pixel displacement between consecutive frames through the optical flow to achieve sub-pixel displacement estimation. When performing KLT optical flow estimation, basic assumptions need to be satisfied: constant brightness, small displacement motion, and spatial consistency assumptions. However, in many practical cases, the small displacement assumption may not hold. For example, when the pixel displacement between consecutive frames is large or the object moves rapidly. In this case, although the error can be reduced through the image pyramid method, this method is not the best choice. Different from the image pyramid method, the image translation transformation method proposed in the video reconstruction effectively satisfies the small displacement assumption and avoids image intensity loss by restricting the structural movement between consecutive frames within one pixel while keeping the image intensity unchanged. The specific steps are as follows: (1) Use the ORB feature detector to detect feature points in the reference frame; (2) Track the movement of each feature point in the current frame according to the spatio-temporal change of the image intensity through the discrete optical flow expression, thereby realizing sub-pixel level displacement estimation. The discrete optical flow expression is where u = dx / dt, v = dy / dt; I x and I y are the partial derivatives of the image intensity at the pixel coordinates (x, y) in the x-direction and y-direction respectively, that is, the image gradient; I tDenote the derivative of the image intensity at pixel coordinates (x, y) with respect to time at time t; ω m is the weight function for each pixel in window W; variables u and v are the optical flows to be determined.

[0030] Step S15: Physical displacement.

[0031] In this embodiment, the physical displacement includes: adding the integer pixel and sub-pixel displacements to obtain an accurate pixel displacement; converting the accurate pixel displacement into a physical displacement through the scale factor to obtain a large displacement of the accurate structural vibration. For example Figure 7 is a flowchart of an accurate physical displacement estimation based on Canny-Hough transform and KLT optical flow disclosed in this application.

[0032] For example Figure 8 is a comparison diagram of the estimated results of the large structural displacement based on Canny-Hough transform and KLT optical flow disclosed in this application. From the recognition results of traditional KLT, pyramid KLT, and the method proposed in this paper, it can be seen that the working performance of traditional KLT optical flow is significantly lower than that of pyramid KLT optical flow. This is because pyramid KLT optical flow attempts to detect large displacement motions through image pyramid downsampling, but there are still large errors in some cases, as shown in node 1. The method proposed in this paper has a high degree of coincidence with the true value, indicating that the method in this paper has good working performance when recognizing large displacement motions of structures. In addition, from the absolute error curve, it can be seen that the method proposed in this paper shows high accuracy in displacement recognition, the absolute value of its error is less than 0.12 mm, and the error distribution is relatively stable. Research shows that a combined method based on Canny-Hough and KLT optical flow, proposed in view of the excellent characteristics of Canny-Hough for pixel-level and KLT optical flow for sub-pixel-level motion recognition, significantly improves the recognition performance of large structural displacements. This combined method effectively overcomes the limitations of single methods in recognizing different pixel levels (pixel level, sub-pixel level), making it more adaptable. In practical applications, this method demonstrates excellent working performance, providing a target-free large displacement measurement method for vibration measurement based on computer vision, and has certain application value.

[0033] It can be seen that the content of this application includes: structural vibration data acquisition and preprocessing; integer pixel displacement estimation based on Canny-Hough transform; video reconstruction; sub-pixel displacement estimation based on KLT optical flow; physical displacement. Thus, this application realizes the recognition of large pixel displacements of structures by restricting pixel-level motions to sub-pixel levels through image translation transformation processing. This processing method avoids image distortion and can effectively guarantee the recognition performance of large displacement motions. This application overcomes the limitations of single recognition methods and significantly improves the measurement accuracy of large structural displacements.

[0034] The above has introduced in detail a large displacement estimation method for structures based on Canny-Hough transform and KLT optical flow. In this article, specific examples are used to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A structural large displacement estimation method based on Canny-Hough transform and KLT optical flow, characterized in that: include: Structural vibration data acquisition and preprocessing; Integer pixel displacement estimation based on Canny-Hough transform; Video reconstruction; Sub-pixel displacement estimation based on KLT optical flow; Physical displacement.

2. The method for estimating large displacement of a structure based on Canny-Hough transform and KLT optical flow according to claim 1, characterized in that: The structural vibration data acquisition and preprocessing includes: Calibrate the acquisition equipment to obtain the internal and external parameters of the camera and the scale factor of the physical displacement to be converted; Collecting original video of structural vibration and correcting it to obtain a corrected vibration video image; Crop and rotate the video to obtain the vibration video of the target area to be identified.

3. The method for estimating large displacement of a structure based on Canny-Hough transform and KLT optical flow according to claim 1, characterized in that: The integer pixel displacement estimation based on Canny-Hough transform includes: Segmenting the vibration video to obtain sequence frame images; Use the Canny operator to detect the edge of the image and obtain the edge features of all frames; Hough transform is used to extract the straight lines and their pixel coordinates in the edge features to achieve displacement estimation at the integer pixel level.

4. The method for estimating large displacement of a structure based on Canny-Hough transform and KLT optical flow according to claim 3, characterized in that: The edge features of all frames are obtained, including: A filter kernel is constructed using a Gaussian function, and a convolution operation is performed on the frame image to achieve smoothing; The expression of Gaussian function is as follows: Where a and b represent the horizontal and vertical coordinates of the image pixels; σ represents the standard deviation of the Gaussian function, which controls the degree of filtering. Perform gradient calculation on the smoothed image to obtain gradient strength and gradient amplitude; The expression for gradient calculation is as follows: In the formula, G x and G y They represent templates for detecting gradient changes in the x and y directions respectively; G and θ represent the gradient strength and direction respectively. Finally, non-maximum elimination and threshold screening operations are performed on the detected gradient intensity map to filter out noise and pixels with small gradients to obtain clearer and more continuous edge features.

5. The method for estimating large displacement of a structure based on Canny-Hough transform and KLT optical flow according to claim 3, characterized in that: The method of realizing displacement estimation at the integer pixel level includes: Extracting straight lines from the edge features by Hough transform; The expression of Hough transform is as follows: ρ=xcosθ+ysinθ Where (ρ, θ) represents the coordinates in the parameter space. After Hough transform, each edge feature point (x, y) in the image space is mapped to a point (ρ, θ) in the parameter space, and then the parameters corresponding to the threshold are found through statistical characteristics. These parameters represent the straight lines in the edge feature map. The pixel coordinates of the straight line in the edge feature map are obtained by extracting the straight line parameters; From the pixel coordinates, an integer pixel displacement is estimated.

6. The method for estimating large displacement of a structure based on Canny-Hough transform and KLT optical flow according to claim 1, characterized in that: The video reconstruction comprises: Calculate the integer pixel displacement between consecutive frames to obtain the integer pixel displacement time history; The image of the target area to be identified is subjected to translation transformation according to the integer pixel displacement time history to reconstruct the video.

7. The method for estimating large displacement of a structure based on Canny-Hough transform and KLT optical flow according to claim 1, characterized in that: The sub-pixel displacement estimation based on KLT optical flow includes: Detecting feature points in the reference frame using a feature detector; Kanade-Lucas-Tomasi (KLT) optical flow is used to track the motion of this feature point in the current frame, thereby estimating the optical flow between consecutive frames; The sub-pixel displacement between consecutive frames is extracted through the optical flow to achieve sub-pixel displacement estimation.

8. The method for estimating large displacement of a structure based on Canny-Hough transform and KLT optical flow according to claim 1, characterized in that: The physical displacement includes: Adding the integer pixel and sub-pixel displacements to obtain an accurate pixel displacement; The precise pixel displacement is converted into a physical displacement by the scaling factor to obtain the precise large displacement of the structural vibration.

Images