A method, system and medium for measuring structural displacement video based on phase cross-correlation

Abstract

The application discloses a structure displacement video measurement method and system based on phase cross-correlation and a medium. Firstly, a multi-feature amplitude is constructed by using video frame gray-scale and morphological dilation technology to provide reliable feature parameters for phase cross-correlation calculation; coarse search is performed by using a phase cross-correlation peak picking strategy, the spatial coordinates of a main peak are acquired, and pixel-level displacement is obtained; fine search is performed by using an extended phase cross-correlation difference strategy, horizontal and vertical decompositions of the cross-correlation function are completed, difference geometric operations are performed on the main peak and a secondary peak, sub-pixel displacement is obtained, and a coarse and fine search mechanism for displacement measurement is formed; an initial frame is used as a reference frame, cross-correlation operation is performed on the current frame, and the conversion from an image coordinate system to a world coordinate system is completed by combining internal and external parameters of camera calibration, and structure motion displacement measurement is accurately realized. The application is simple in operation and flexible in method, displacement measurement is completed by using the shape or natural texture of the structure, and dense and high-precision measurement results are realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of structural displacement measurement, and particularly relates to a structural displacement video measurement method and system based on phase cross-correlation and a medium. BACKGROUND

[0002] In the fields of bridge construction, aerospace and mechanical engineering, structural displacement measurement and analysis always have important practical significance. As an important indicator of overall structural behavior and safety, displacement helps to understand the structural characteristics and assess the structural safety by monitoring the dynamic displacement of the structure under different load types. Structural displacement measurement technology is divided into contact and non-contact types. Traditional contact technology measures displacement by installing sensors on the structure surface, which has the disadvantages of complex wiring, difficult installation, low spatial resolution, etc. Meanwhile, the installation of sensors will bring quality loading effect, which may change the dynamics characteristics of the structure, leading to the decline of the structural performance and affecting the accuracy and objectivity of the test results. The non-contact laser triangulation sensor has the problems of short measurement range and high price, while the visual measurement technology combined with image processing algorithm has the significant advantages of full-field measurement, remote measurement, non-contact, low cost, flexible use, etc., and has become an important part of the field of non-contact structural displacement measurement technology, with broad application prospects.

[0003] In order to fully exert the significant advantages of visual measurement method, the current feature-based point tracking method and the template matching method based on digital image correlation (DIC) provide a new perspective for structural displacement measurement technology and have been widely developed and applied. The patent with publication number CN118376162A proposes a bridge structure displacement influence line measurement method based on computer vision technology, which uses corner point detection method to identify the feature points of the bridge structure and extracts the feature point coordinates to track the displacement of the bridge structure. However, the point tracking technology has challenges in the number of key points extracted in the given region of interest (ROI), and the measurement results are usually sparse, which cannot meet the high resolution requirement of displacement measurement. In addition, multiple parameters and thresholds need to be adjusted in the measurement process. The patent with publication number CN117392368A proposes a region of interest setting method and system for structural displacement measurement, which uses DIC-based template matching and evaluates the feature points of ROI to improve the number of ROI feature point detection and matching, meeting the high-precision structural deformation measurement requirement. However, the template matching method has the disadvantage of time-consuming calculation, and usually needs to pre-set artificial markers on the structure to be measured. SUMMARY

[0004] The present application aims to provide a structural displacement video measurement method and system based on phase cross-correlation and a medium.

[0005] The present application achieves the above-mentioned purpose by the following technical solutions:

[0006] A phase cross-correlation-based structure displacement video measurement method, the specific steps are as follows:

[0007] Step 1: using a stable light source to irradiate the structure to be measured, using a video acquisition device to record the vibration video of the structure, and performing gray processing on the vibration video of the structure to be measured;

[0008] Step 2: performing multi-feature processing on the gray video of the structure to be measured to obtain multi-feature amplitudes with more shape features than original video frames;

[0009] Step 3: selecting a region of interest in the multi-feature amplitude frame sequence, and using a phase cross-correlation peak picking strategy to complete pixel-level displacement measurement;

[0010] Step 4: based on the phase cross-correlation peak picking strategy, the cross-correlation function is decomposed into one-dimensional signals in the horizontal and vertical directions; and then using an extended phase cross-correlation difference strategy to complete sub-pixel displacement measurement in the horizontal and vertical directions;

[0011] Step 5: using the initial frame as a reference frame, and performing correlation operation with the subsequent frames, that is, repeating steps 3-4, to realize fast and accurate measurement of the vibration displacement of the measured object, and generate a vibration displacement signal;

[0012] Step 6: finally, using the internal and external parameters of camera calibration, completing the conversion from the image coordinate system to the world coordinate system, and completing the real measurement of the vibration displacement and the modal analysis.

[0013] Further, the structure to be measured in step 1 has edge features or natural textures; the original RGB video is grayed using the formula Gray=R×0.299+G×0.587+B×0.114, and R, G and B are respectively the three color channel values of the image.

[0014] Further, the multi-feature processing of step 2 includes three steps of image morphological dilation, complex amplitude model construction and multi-feature amplitude extraction;

[0015] Step 2.1: performing morphological processing on the gray video frame, and performing image dilation; using a structure element K to dilate an input image I(x, y), and the calculation formula of the dilated image I D (x, y) is:

[0016]

[0017] Where x and y represent the position of the pixel on the camera sensor plane, and ⊕ represents the morphological dilation operation.

[0018] Step 2.2: The complex amplitude model is constructed by using the original image and the dilated image as the real part and the imaginary part of the complex analytic signal respectively, and the construction formula is:

[0019] I DA (x,y)=I(x,y)+iI D (x,y)

[0020] Step 2.3: The modulus of the constructed complex analytic signal is calculated to obtain the multi-feature amplitude of the original video frame, and abs[·] represents the modulus operation, and the calculation formula is:

[0021] T(x,y)=abs[I DA (x,y)]

[0022] Further, the step 3 utilizes the phase cross-correlation peak picking strategy to measure the pixel-level displacement signal of the structure to be measured, specifically:

[0023] Step 3.1: Select the region of interest ROI from the multi-feature amplitude sequence after the gray video dilatation processing;

[0024] Step 3.2: Perform Fourier transform on the multi-feature amplitude ROI, and obtain the two-dimensional sinc function c(x,y) according to the inverse Fourier transform of the normalized cross power spectrum, and F represents the Fourier transform and * represents the conjugate operation, and the calculation formula is:

[0025]

[0026] Step 3.3: The main peak Max_peak(δ x ,δ y ) is obtained by using the peak picking method on the c(x,y), and its coordinates (δ x ,δ y ) are recorded, and the pixel-level displacement measurement is completed, and max[·] represents the maximum value operation, and the calculation formula is:

[0027] Max_peak(δ x ,δ y )=max[c(x,y)].

[0028] Further, the step 4 utilizes the extended phase cross-correlation difference strategy to measure the sub-pixel displacement signal of the structure to be measured in the horizontal and vertical directions, specifically:

[0029] Step 4.1: The main peak coordinates of the two-dimensional sinc function c(x,y) in step 3 are taken as the center to perform horizontal and vertical direction decomposition to generate corresponding one-dimensional signals c(x) and c(y);

[0030] Step 4.2: Pick the main peak and the sub-peak of the one-dimensional signals c(x) and c(y) respectively, and perform the difference geometry operation to obtain the sub-pixel displacement with strong anti-noise, and complete the displacement measurement at the sub-pixel level; use x main , x sec and x third to represent the main peak and the sub-peak coordinates of c(x) respectively, and the horizontal direction displacement analytical expression is as follows, and the vertical direction is the same.

[0031]

[0032] A measurement system of a structure displacement video measurement method based on phase cross-correlation, comprising an acquisition module, an expansion module, a coarse measurement module, a fine measurement module and an output module.

[0033] The acquisition module comprises an LED light source, a high-speed camera, a camera lens, a tripod and a computer, and is used for acquiring the vibration video of the structure to be measured in step 1.

[0034] The expansion module is used for performing gray scale processing on the structure vibration video in step 1, and performing expansion construction processing on the structure vibration video to obtain a plurality of characteristic amplitudes in step 2.

[0035] The coarse measurement module is used for selecting a region of interest from the video after the gray scale expansion processing in step 3, and using a phase cross-correlation peak picking strategy to locate the maximum peak coordinates of the two-dimensional Sine function, and then measuring the pixel-level displacement signal of the structure vibration to be measured.

[0036] The fine measurement module is used for using the two-dimensional Sine function as input in step 4, and converting it into a one-dimensional signal, and using an extended phase cross-correlation difference strategy to measure the sub-pixel displacement signal of the structure vibration to be measured in the horizontal and vertical directions.

[0037] The output module is used for performing the expansion module, the coarse measurement module and the fine measurement module on all video frames and the initial frame in steps 5 and 6, quickly obtaining accurate and robust displacement signals, and completing real measurement of the structure displacement and modal analysis through conversion from the image coordinate system to the world coordinate system.

[0038] A computer readable storage medium having a computer program / instruction stored thereon, the computer program / instruction being executed by a processor to implement the steps of a structure displacement video measurement method based on phase cross-correlation.

[0039] A computer program product comprising a computer program / instruction, the computer program / instruction being executed by a processor to implement the steps of a structure displacement video measurement method based on phase cross-correlation.

[0040] An electronic device comprises a memory for saving a computer program, and a processor for executing the computer program to realize an instruction tracking method based on a phase cross-correlation structural displacement video measurement method.

[0041] The present application has the following advantages:

[0042] 1. Compared with a feature-based point tracking method, the present application is based on phase cross-correlation matching, has dense and accurate high-resolution measurement results, and has good robustness to environmental noise by combining a phase cross-correlation peak picking strategy with an extended phase cross-correlation differential strategy, and can still show excellent measurement results in a noisy video.

[0043] 2. Compared with a DIC-based template matching method, the present application uses morphological dilation technology to increase structural features, thereby using the shape or natural texture of the structure to replace artificial markers, and has a simple operation mode and flexible measurement mode. By using frequency domain phase spectrum correlation matching, only a single matching is required to realize fast measurement of structural displacement, effectively improving the measurement efficiency and making real-time measurement of structural displacement possible. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 which is a schematic diagram of the overall workflow of the present application;

[0045] Figure 2 which is a schematic diagram of the multi-feature amplitude extraction method of the present application;

[0046] Figure 3 which is a schematic diagram of the phase cross-correlation peak picking strategy of the present application;

[0047] Figure 4 which is a schematic diagram of the extended phase cross-correlation differential strategy of the present application;

[0048] Figure 5 which is a simulation picture of an under-damped oscillating spherical ball;

[0049] Figure 6 which is a displacement measurement result of the present application;

[0050] Figure 7 which is a picture of a cantilever beam in the actual camera plane;

[0051] Figure 8 which is an actual measurement result of the present application. DETAILED DESCRIPTION

[0052] The present application will be further described below with reference to the accompanying drawings.

[0053] Reference Figures 1 to 4The embodiment provides a structure displacement video measurement method based on phase cross correlation. The technical solutions in the embodiments of the present application will be clearly and completely described in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0054] The hardware used by the structure displacement video measurement method based on phase cross correlation of the present application comprises an LED light source, a high-speed camera, a camera lens, a tripod and a computer. Before testing, monocular camera calibration is performed, which is a one-time work, and the purpose is to obtain the conversion coefficient of the image coordinate system and the world coordinate system by using the monocular camera calibration principle. The conversion coefficient is obtained according to the internal parameters of the camera and the external parameters between the camera and the moving structure. The internal parameters mainly comprise the focal length and the pixel size of the camera, and the external parameters mainly comprise the object distance.

[0055] When working, as shown in Figure 1 the structure displacement video measurement method based on phase cross correlation of the present application comprises the following steps:

[0056] Step 1: a stable light source is used to irradiate a structure to be measured, the structure to be measured should have edge features or natural textures, and a video acquisition device is used to record vibration of the structure;

[0057] Step 2: vibration video of the structure to be measured is acquired, and the vibration video is subjected to grayscale processing, the original RGB video is grayscaled by using a formula Gray=R*0.299+G*0.587+B*0.114, and R, G and B are respectively three color channel values of the image;

[0058] Step 3: the grayscale video of the structure to be measured is subjected to multi-feature processing, including three steps of image morphological dilation, complex amplitude model construction and multi-feature amplitude extraction, and multi-feature amplitudes with more shape features than original video frames are acquired, as shown in Figure 2 , and the specific steps are as follows:

[0059] (1) the grayscale video frame is subjected to morphological processing, and image dilation is performed; the input image I(x, y) is dilated by using a structure element K, and the calculation formula of the dilated image I D (x, y) is as follows:

[0060]

[0061] wherein x and y represent the positions of pixels on the camera sensor plane, represents a morphological dilation operation;

[0062] (2) The original image and the dilated image are used as the real part and the imaginary part of the complex analytic signal respectively, and the formula for constructing the complex amplitude model is:

[0063] I DA (x,y) = I(x,y) + iI D (x,y)

[0064] (3) The modulus of the constructed complex analytic signal is calculated to obtain the multi-feature amplitude of the original video frame, and abs[·] represents the modulus operation, and the calculation formula is:

[0065] T(x,y) = abs[I DA (x,y)]

[0066] Step 4: Select the region of interest from the multi-feature amplitude frame sequence, and use the phase cross-correlation peak picking strategy to complete the pixel-level displacement measurement, as shown in Figure 3 , specifically:

[0067] (1) Select the region of interest ROI from the multi-feature amplitude sequence after the grayscale video dilatation processing;

[0068] (2) Perform Fourier transform on the multi-feature amplitude ROI, and obtain the two-dimensional sinc function c(x,y) according to the inverse Fourier transform of the normalized cross power spectrum, where represents the Fourier transform, and * represents the conjugate operation, and the calculation formula is:

[0069]

[0070] (3) Use the peak picking method on the c(x,y) to obtain the main peak Max_peak(δ x ,δ y ) and record its coordinates (δ x ,δ y ), and then complete the pixel-level displacement measurement; max[·] represents the maximum value operation, and the calculation formula is:

[0071] Max_peak(δ x ,δ y ) = max[c(x,y)]

[0072] Step 5: Based on the phase cross-correlation peak picking strategy, the cross-correlation function is decomposed into one-dimensional signals in the horizontal and vertical directions. The main peak coordinates of the two-dimensional sinc function c(x,y) are taken as the center to perform horizontal and vertical direction decomposition, and the corresponding one-dimensional signals c(x) and c(y) are generated;

[0073] Step 6: The horizontal and vertical sub-pixel displacement measurement is quickly, accurately and robustly completed by using the extended phase cross-correlation differential strategy. The main peak and the secondary peak of the one-dimensional signals c(x) and c(y) are picked up respectively, and the differential geometric operation is performed to complete the sub-pixel level displacement measurement, as shown in Figure 4 main sec third The horizontal direction displacement analytical expression is as follows, and the vertical direction is the same.

[0074]

[0075] Step 7: The initial frame is used as a reference frame, and the correlation operation is performed with the subsequent frames, that is, steps 4-6 are repeated, so that the vibration displacement of the measured member is quickly and accurately measured, and the vibration displacement signal is generated;

[0076] Step 8: The conversion factor obtained by using the internal and external parameters of the camera is used to complete the conversion from the image coordinate system to the world coordinate system, and the real measurement and modal analysis of the vibration displacement are completed.

[0077] The application also provides a structure displacement video measurement system based on phase cross-correlation, which comprises:

[0078] Module 1: Acquisition module, which acquires the vibration video of the measured structure according to the content of step 1;

[0079] Module 2: Expansion module, which is used for pre-processing the structure vibration video according to the content of steps 2 and 3, including video graying and expansion construction processing, and acquiring the multi-feature amplitude;

[0080] Module 3: Coarse measurement module, which is used for selecting the region of interest from the video after the graying and expansion processing according to the content of step 4, locating the maximum peak coordinates of the two-dimensional sinc function by using the phase cross-correlation peak picking strategy, and then measuring the pixel level displacement signal of the vibration of the measured structure;

[0081] Module 4: Fine measurement module, which is used for taking the two-dimensional sinc function of the coarse measurement module as input according to the content of steps 5 and 6, taking it as a one-dimensional signal respectively, and measuring the sub-pixel displacement signal of the vibration of the measured structure in the horizontal and vertical directions by using the extended phase cross-correlation differential strategy;

[0082] Module 5: Output module, which performs the expansion module, the coarse measurement module and the fine measurement module on all video frames and the initial frame according to the content of steps 7 and 8, quickly obtains the accurate and robust displacement signal, and completes the real measurement and modal analysis of the structure displacement through the conversion from the image coordinate system to the world coordinate system.

[0083] ​​​In the first embodiment, a structure displacement video measurement method based on phase cross-correlation is provided, and a step flow chart is shown in Figures 1-4 The simulation generates a circular sphere with a radius of 128 pixels, and the sphere is subjected to under-damped vibration in the horizontal direction, wherein the amplitude is 10 pixels, the attenuation factor is 1 / 30, the frame rate is set to 200, and a video with a time length of 1 second is generated, as shown in Figure 5 A linear dilation operator is selected, which is generally universal. The multi-feature amplitude is constructed frame by frame from the video frame sequence. The pixel-level displacement measurement is completed by using the phase cross-correlation peak picking strategy, and the sub-pixel displacement measurement is completed by using the extended phase cross-correlation difference strategy. Since the simulation sample does not involve the conversion factor of the world-image coordinate system, the actual motion field is directly obtained. The time domain measurement result is shown in Figure 6 The error between the simulation setting data and the calculation data of the application is compared, and the RMSE calculation result is 0.028 pixels.

[0084] In the second embodiment, a structure displacement video measurement method based on phase cross-correlation is provided, and a step flow chart is shown in Figures 1-4 The sample preparation and measurement results of the embodiment are as follows: a cantilever beam with a length of 50 cm is prepared, and the bottom of the cantilever beam is excited by using a force hammer. The vibration process of the cantilever beam is collected and recorded by using a digital camera or a mobile phone to obtain a video containing the vibration information of the cantilever beam, and a contact method (triangular light laser sensor measurement) is used to measure the vibration of the measured object as a comparative reference value. During the collection process, the frame rate is set to 1000, and an LED light source is used for lighting, and the overall structure is shown in Figure 7 A linear dilation operator is selected, which is generally universal. The multi-feature amplitude is constructed frame by frame from the video frame sequence. The pixel-level displacement measurement is completed by using the phase cross-correlation peak picking strategy, and the sub-pixel displacement measurement is completed by using the extended phase cross-correlation difference strategy. The conversion factor of the actual displacement of the measured object in the actual space plane motion and the motion size (pixels) of the actual measured object in the camera plane is 0.2248 mm / pixel. The actual motion field is obtained by multiplying the time sequence absolute motion displacement field by the conversion proportion factor, and the time domain measurement result is shown in Figure 8 The error between the actual measurement data and the calculation data of the application is compared, and the RMSE calculation result is 0.0164 mm.

[0085] The above only describes the preferred embodiments of the application and is not used to limit the application. For those skilled in the art, the application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A method for measuring structural displacement using video based on phase cross-correlation, characterized by: The specific steps are as follows: Step 1: Use a stable light source to illuminate the structure to be tested, use a video acquisition device to record the vibration video of the structure to be tested, and grayscale the vibration video of the structure to be tested; Step 2: Perform multi-feature processing on the grayscale video of the structure to be tested; Step 2.1: Perform morphological processing on the grayscale video frame and perform image expansion; use the structure element K to expand an input image I (x, y), and expand the image I D The formula for (x,y) is: Where x and y represent the position of the pixel on the camera sensor plane, represents the morphological dilation operation; Step 2.2: Using the original image and the expanded image as the real and imaginary parts of the complex analytical signal respectively, the complex amplitude model is constructed as follows: I DA (x,y)=I(x,y)+iI D (x,y) Step 2.3: Calculate the modulus of the constructed complex analytic signal to obtain the multi-feature amplitude of the original video frame. Let abs[·] represent the modulus operation, and its calculation formula is: T(x,y)=abs[I DA (x,y)]; Step 3: Use the phase cross-correlation peak picking strategy to complete pixel-level displacement measurement; Step 3.1: selecting a region of interest (ROI) from the multi-feature amplitude sequence after the grayscale video dilation processing; Step 3.2: Perform Fourier transform on the multi-feature amplitude ROI, and obtain the two-dimensional Single function c(x, y) based on the inverse Fourier transform of the normalized cross power spectrum. Let F represent Fourier transform and * represent conjugate operation. The calculation formula is: Step 3.3: Use the peak picking method to obtain the main peak Max_peak (δ x ,δ y ) and record its coordinates (δ x ,δ y ), and then complete the pixel-level displacement measurement, let max[·] represent the maximum value operation, and its calculation formula is: Max_peak(δ x ,δ y )=max[c(x,y)]; Step 4: Based on the phase cross-correlation peak picking strategy, the cross-correlation function is decomposed into one-dimensional signals in the horizontal and vertical directions; and then the extended phase cross-correlation difference strategy is used to complete the sub-pixel displacement measurement in the horizontal and vertical directions; Step 5: Use the initial frame as a reference frame and perform correlation operations with subsequent frames, i.e. repeat steps 3-4 to measure the vibration displacement of the test piece and generate a vibration displacement signal; Step 6: Finally, use the internal and external parameters of the camera calibration to complete the conversion from the image coordinate system to the world coordinate system, and complete the actual measurement of vibration displacement and modal analysis.

2. The method for measuring structural displacement using video based on phase cross-correlation according to claim 1, wherein: In step 1, the structure to be measured has edge features or natural textures; the vibration video grayscales the original RGB video using the formula Gray=R×0.299+G×0.587+B×0.114, where R, G, and B are the three color channel values ​​of the image respectively.

3. The method for measuring structural displacement using video based on phase cross-correlation according to claim 1, wherein: The step 4 uses the extended phase cross-correlation difference strategy to measure the sub-pixel displacement signals of the vibration of the structure to be measured in the horizontal and vertical directions, specifically: Step 4.1: Decompose the two-dimensional Singer function c(x,y) described in step 3 horizontally and vertically, with the main peak coordinates as the center, to generate the corresponding one-dimensional signals c(x) and c(y); Step 4.2: Pick the main peak and secondary peak of the one-dimensional signal c(x) and c(y) respectively, and perform differential geometric operations to obtain the sub-pixel displacement with strong noise resistance, and complete the sub-pixel displacement measurement; use x main 、x sec with x third Represent the coordinates of the main peak and secondary peak of c(x), respectively. The analytical expression of the horizontal displacement is as follows, and the vertical displacement is the same; 4. A measurement system for a structural displacement video measurement method based on phase cross-correlation according to any one of claims 1 to 3, characterized in that: It includes acquisition module, expansion module, rough measurement module, fine measurement module and output module; The acquisition module includes an LED light source, a high-speed camera, a camera lens, a tripod, and a computer; and is used to obtain the vibration video of the structure to be measured in step 1; The expansion module is used to grayscale the structural vibration video in step 1; and to perform expansion and structural processing on the structural vibration video in step 2 to obtain multiple characteristic amplitudes; The coarse measurement module is used to select a region of interest from the video after the grayscale expansion process in step 3, locate the maximum peak coordinates of the two-dimensional Singer function using a phase cross-correlation peak picking strategy, and then measure the pixel-level displacement signal of the vibration of the structure to be measured; The precision measurement module is used to use the two-dimensional Singer function as input in step 4, decompose it into a one-dimensional signal, and use the extended phase cross-correlation difference strategy to measure the sub-pixel displacement signals of the vibration of the structure to be measured in the horizontal and vertical directions; The output module is used in steps 5 and 6 to perform the expansion module, coarse measurement module, and fine measurement module operations on all video frames and the initial frame to obtain displacement signals, and complete the real-world measurement and modal analysis of the structural displacement by converting the image coordinate system to the world coordinate system.

5. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

6. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

7. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the method according to any one of claims 1 to 3.

Images