A method for measuring structural motion optical flow based on pre-sequence motion sequence analysis

Abstract

The application discloses a structure motion optical flow measurement method based on pre-sequence motion sequence analysis. Firstly, the motion video of a measured structure is collected, and each frame of the video is subjected to grayscale and Gaussian smoothing denoising treatment. Secondly, the corner point characteristics are screened through a corner point response factor, a pixel point set with measurement robustness is established in a structure imaging area, and a measurement point is selected from the pixel point set. Then, the window size is initialized, the number of pre-sequence reference frames is set, the motion of the measurement point in all pre-sequence reference frames is calculated to obtain the pre-sequence motion sequence. Finally, a sequence analysis model is established, the motion of the measurement point in the next frame is calculated based on the pre-sequence motion sequence to predict the optical flow matching window size of the next frame, the motion of the measurement point in the next frame is added to the pre-sequence motion sequence, and the motion of the earliest frame in the original sequence is removed, and the process is repeated until the last frame. The application effectively utilizes the statistical information of the structure motion history in the time domain, overcomes the problem that a static window cannot simultaneously consider the calculation stability and the measurement accuracy, and can realize high-precision optical flow measurement of the structure motion based on vision.

Description

Technical Field

[0001] This invention belongs to the field of mechanical equipment health monitoring technology, and specifically relates to a method for measuring structural motion optical flow based on preceding motion sequence analysis. Background Technology

[0002] Structural motion measurement is a crucial task in the health monitoring of mechanical equipment, reflecting the dynamic characteristics of a structure under working conditions. Due to its significant advantages such as non-contact operation, high spatial resolution, and portability, vision-based structural motion measurement methods have gradually gained more attention in industry. Currently, various vision algorithms have been developed, including template matching, feature tracking, digital image correlation, and optical flow algorithms. Among them, the optical flow algorithm is a vision algorithm that reflects structural motion by analyzing the temporal changes in image pixel grayscale. Compared to other vision methods, the optical flow algorithm does not require explicit tracking of structural surface features, does not rely on special manual markings on the structural surface, and boasts high computational efficiency, enabling real-time measurement and making it more suitable for practical engineering tasks.

[0003] However, when performing structural motion measurements, optical flow algorithms require prior knowledge of neighborhood motion consistency. This involves selecting an optical flow matching window centered on the measurement point and solving for the optical flow at that center using least squares. The size of the optical flow matching window significantly impacts the measurement results. A larger window size improves computational stability for fast motion but reduces accuracy for slow motion; conversely, a smaller window size improves accuracy for slow motion at the expense of computational stability for fast motion. Currently, the selection of the optical flow matching window size remains a critical issue, especially when dealing with highly irregular structural motion, where the performance of optical flow algorithms cannot be guaranteed. Summary of the Invention

[0004] To overcome the shortcomings of existing optical flow algorithm-based structural motion measurement methods and address the limitation of current optical flow algorithm performance by the optical flow matching window size, this invention aims to propose a structural motion optical flow measurement method based on prior motion sequence analysis. This method effectively utilizes historical statistical information of the structural motion sequence, overcomes the problem that static windows cannot simultaneously ensure computational stability and measurement accuracy, and enables high-precision optical flow measurement of structural motion.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for measuring structural motion optical flow based on preceding motion sequence analysis includes the following steps:

[0007] Step 1: Use a high-speed camera to capture motion video of the structure under test, and perform grayscale conversion and Gaussian smoothing noise reduction on each frame of the video;

[0008] Step 2: Filter pixel corner characteristics by corner response factor, establish a set of pixels with measurement robustness in the structural imaging area, and select measurement points from them;

[0009] Step 3: Initialize the window size and set the number of preceding reference frames. Calculate the optical flow by minimizing the optical flow matching error function, and obtain the motion of the measurement point across all preceding reference frames as the preceding motion sequence.

[0010] Step 4: Establish a structural motion sequence analysis model, predict the optical flow matching window size of the next frame based on the previous motion sequence, calculate the motion of the measurement point in the next frame, add it to the previous motion sequence and remove the motion of the earliest frame in the original sequence, and repeat this process until the last frame.

[0011] Compared with existing structural motion measurement methods based on optical flow algorithms, the advantages of this invention are as follows:

[0012] (1) This invention automatically filters the corner characteristics of all pixels by pre-calculating the corner response factor, thereby providing a highly robust measurement point that can be used for optical flow calculation, avoiding the blind selection of measurement points, and eliminating the need for special manual marking on the structural surface.

[0013] (2) This invention effectively utilizes the historical statistical information of the structural motion sequence, and can dynamically predict and update the window size in real time. It overcomes the problem that static windows cannot take into account both computational stability and measurement accuracy, and can realize high-precision optical flow measurement of structural motion. Attached Figure Description

[0014] Figure 1 This is a flowchart of a structure motion optical flow measurement algorithm based on preceding motion sequence analysis.

[0015] Figure 2 This is a schematic diagram showing the movement of the optical flow matching window between frames.

[0016] Figure 3 This is a schematic diagram illustrating the real-time dynamic window update based on structural motion sequence analysis according to the present invention.

[0017] Figure 4 (a) is the time-domain information of motion of a concrete structure measured using the method of the present invention.

[0018] Figure 4 (b) is the history of dynamic window size adjustment during the measurement process of the method of the present invention. Detailed Implementation

[0019] The present invention will now be further described with reference to the accompanying drawings.

[0020] like Figure 1 As shown, a method for measuring structural motion optical flow based on preceding motion sequence analysis includes the following steps:

[0021] Step 1: Use a high-speed camera to capture motion video of the structure under test, and perform grayscale conversion and Gaussian smoothing noise reduction on each frame of the video;

[0022] Step 2: Filter pixel corner characteristics by corner response factor, establish a set of pixels with measurement robustness in the structural imaging area, and select measurement points from them;

[0023] First, the corner response factor is calculated to quantify the pixel corner characteristics of the initial frame of the video. The corner response matrix of each pixel can be calculated as follows:

[0024]

[0025] In the formula, H(x,y) is the corner response matrix of the pixel at coordinates (x,y), and I x (m,n) and I y (m,n) are the gray-level gradients of the pixel at coordinates (m,n) in the horizontal and vertical directions, respectively, and W(x,y) is the neighborhood window of the pixel at coordinates (x,y). (m,n) are all located within the set W(x,y).

[0026] When the pixel corner characteristics meet the measurement criteria, the two eigenvalues ​​of the corner response matrix should be at local peaks. Therefore, the corner response factor is defined as:

[0027]

[0028] In the formula, Ind(x,y) is the corner response factor of the pixel at coordinates (x,y), and λ1(x,y) and λ2(x,y) are two eigenvalues ​​of the corner response matrix H(x,y), respectively.

[0029] After calculating the corner response factors of all pixels, sort them and use the median corner response factor Ind as the index. mid As a screening threshold for pixel corner characteristics, a set E of pixels with high measurement robustness is obtained, and measurement points are selected from it:

[0030] E = {(x,y)Ind(x,y)>Ind} mid |(x,y)∈R}

[0031] In the formula, R is the structural imaging region;

[0032] Step 3: Initialize the window size and set the number of preceding reference frames. Calculate the optical flow by minimizing the optical flow matching error function, and obtain the motion of the measurement point across all preceding reference frames as the preceding motion sequence.

[0033] Let the initial window dimensions be as follows: The preceding reference frame number is n. First, based on the initially set window size, an optical flow algorithm is used to calculate the motion of the measurement point in the first 2n+1 frames, such as... Figure 2 The diagram illustrates the movement of the optical flow matching window between frames. When the measurement point moves between adjacent frames, the optical flow matching error function is defined as:

[0034]

[0035] In the formula, ε is the optical flow matching error function, and p = (p x ,p y ) represents the coordinates of the measurement point in the current frame, d = (d x ,d y Let I(x,y) be the optical flow of the measurement point in the current frame, and let J(x+d) be the pixel gray level at coordinates (x,y) in the current frame. x ,y+d y ) represents the coordinates of the next frame in (x+d) x ,y+d y The pixel grayscale at position ) is ωx and ωy, which are the half-window sizes in the x and y directions, respectively.

[0036] By minimizing the optical flow matching error function, the optical flow can be obtained as follows:

[0037]

[0038] The optical flow between all preceding reference frames is calculated, and the motion of the measurement point in the first 2n+1 frames is obtained as the preceding motion sequence:

[0039] s={d 1 ,d 2 ,...,d 2n+1}

[0040] In the formula, s is the preceding motion sequence, and d i The optical flow at the measurement point in the i-th frame;

[0041] Step 4: Establish a structural motion sequence analysis model, predict the optical flow matching window size of the next frame based on the previous motion sequence, calculate the motion of the measurement point in the next frame, add it to the previous motion sequence and remove the motion of the earliest frame in the original sequence, and repeat this process until the last frame.

[0042] Based on the assumption of temporal dependence, that is, there is a strong correlation between structural motions in different frames, and the structural motion of a given frame is significantly influenced by its past structural motions, a structural motion sequence analysis model is established, as shown in the following equation: the structural motion of the next frame can be predicted from its preceding motions.

[0043]

[0044] In the formula, Let a0 be the predicted motion for frame t, and a be a constant term. i This is the influence factor of the preceding motion of the past i frames on the motion of the current frame;

[0045] By substituting the preceding motion sequence data into the established structural motion sequence analysis model, the following equations can be obtained:

[0046]

[0047] Therefore, we can perform operations on a0, a1, ..., a n Solve the problem. For example... Figure 3 The diagram illustrates real-time dynamic window updates based on structural motion sequence analysis. The structural motion sequence analysis model predicts the motion of the next frame. The optical flow matching window size is updated as a reference value for the next frame, as shown in the following formula:

[0048]

[0049] Then (2w) t +1) is the size of the optical flow matching window in the current t-th frame;

[0050] Based on the optical flow matching window size, the optical flow algorithm is used to calculate the motion of the measurement point in the next frame, add it to the previous motion sequence and remove the motion of the earliest frame in the original sequence, repeating until the last frame of the video, and obtaining the optical flow of the measurement point in all frames, thereby obtaining high-precision temporal information of structural motion.

[0051] The method of this invention is used to perform motion measurement on concrete building structures. The measurement results of the time-domain information of structural motion are as follows: Figure 4 As shown in (a), the dynamic window adjustment history during the measurement process is as follows: Figure 4 As shown in (b), the method of the present invention can predict and adjust the matching window size in real time according to the structural motion, taking into account both computational stability and measurement accuracy, and is suitable for structural motion measurement under actual complex working conditions.

Claims

1. A method for measuring structural motion optical flow based on prior motion sequence analysis, characterized in that, Includes the following steps: Step 1: Use a high-speed camera to capture motion video of the structure under test, and perform grayscale conversion and Gaussian smoothing noise reduction on each frame of the video; Step 2: Filter pixel corner characteristics by corner response factor, establish a set of pixels with measurement robustness in the structural imaging area, and select measurement points from them; Step 3: Initialize the window size and set the number of preceding reference frames. Calculate the optical flow by minimizing the optical flow matching error function, and obtain the motion of the measurement point across all preceding reference frames as the preceding motion sequence. Step 4: Establish a structural motion sequence analysis model, predict the optical flow matching window size of the next frame based on the previous motion sequence, calculate the motion of the measurement point in the next frame, add it to the previous motion sequence and remove the motion of the earliest frame in the original sequence, and repeat this process until the last frame. Step 4 is as follows: Based on the assumption of temporal dependence, that is, there is a strong correlation between structural motions in different frames, and the structural motion of a given frame is significantly influenced by its past structural motions, a structural motion sequence analysis model is established, as shown in the following equation: the structural motion of the next frame is predicted by its preceding motions. In the formula, For the first t Predicted motion of frames, a 0 is a constant term. a i For the past i The influence factor of the preceding motion of a frame on the motion of the current frame; By substituting the preceding motion sequence data into the established structural motion sequence analysis model, the following equations are obtained: Therefore, it is possible to a 0, a 1, …, a n The solution is performed, and the motion of the next frame is predicted based on the structural motion sequence analysis model. As a reference value for the optical flow matching window size in the next frame, it is shown in the following formula: but For the current number t The optical flow matching window size of the frame; Based on the optical flow matching window size, the optical flow algorithm is used to calculate the motion of the measurement point in the next frame, add it to the previous motion sequence and remove the motion of the earliest frame in the original sequence, repeating until the last frame of the video, and obtaining the optical flow of the measurement point in all frames, thereby obtaining high-precision temporal information of structural motion.

2. The structural motion optical flow measurement method based on prior motion sequence analysis according to claim 1, characterized in that, In step 2, the pixel corner characteristics are screened using corner response factors to establish a set of pixels with measurement robustness in the structural imaging region, and measurement points are selected from this set, as follows: First, the corner response factor is calculated to quantify the pixel corner characteristics of the initial frame of the video: the corner response matrix of each pixel is as follows: In the formula, For coordinates at The corner response matrix of the pixel. and The coordinates are respectively in The grayscale gradient of the pixel at that location in the horizontal and vertical coordinate directions. For coordinates at The neighborhood window of the pixel at that location. All located in Within the set; When the pixel corner characteristics meet the measurement criteria, the two eigenvalues ​​of the corner response matrix should be at local peaks. Therefore, the corner response factor is defined as: In the formula, For coordinates at The corner response factor of the pixel at that location. and Corner response matrices Two eigenvalues; After calculating the corner response factors of all pixels, sort them and use the median corner response factor as the criterion. Using pixel corner characteristics as a screening threshold, a set of pixels with high measurement robustness is obtained. E And select measurement points from them: In the formula, R This is the area for structural imaging.

3. The structural motion optical flow measurement method based on prior motion sequence analysis according to claim 1, characterized in that, In step 3, the window size is initialized, and the number of preceding reference frames is set. The optical flow is calculated by minimizing the optical flow matching error function, and the motion of the measurement point in all preceding reference frames is obtained as the preceding motion sequence, as follows: Let the initial window dimensions be as follows: , The preceding reference frame number is n First, based on the initially set window size, the optical flow algorithm is used to calculate the measurement points in the first two... n +1 frame of motion; when the measurement point moves between two adjacent frames, the optical flow matching error function is defined as: In the formula, Let be the optical flow matching error function. The coordinates of the measurement point in the current frame. For measuring the optical flow of the point in the current frame, The current frame coordinates are in pixel grayscale at that location, For the coordinates of the next frame The pixel grayscale at that location, ω x and ω y They are respectively x , y The size of the half-window in the direction; By minimizing the optical flow matching error function, the optical flow is obtained as follows: In the formula, For measurement points The flow of light at that location, The current frame coordinates are in pixel grayscale at that location, For the coordinates of the next frame pixel grayscale at that location, and The current frame is used as the coordinate in The grayscale gradient of the pixel at that location in the horizontal and vertical coordinate directions; The optical flow between all preceding reference frames is calculated, and the measurement points are obtained in the first two frames. n After the motion of +1 frame, it serves as the preceding motion sequence: In the formula, s is the preceding motion sequence. d i For the measurement point at the th i Optical flow of a frame.

Images