Non-contact dynamic measurement method and system for structural deformation based on super-sensitivity optical flow method, processing device, and storage medium

Abstract

A non-contact dynamic measurement method and system for structural deformation based on a super-sensitivity optical flow method, a processing device, and a storage medium. The method comprises: acquiring an image of a structure to be measured; calibrating a scale parameter on the basis of the physical dimension of the structure to be measured and the number of pixels occupied by a structural image; using pixels having large grayscale gradients from N t image frames to construct an effective pixel grayscale spatiotemporal matrix; performing singular value decomposition on the effective pixel grayscale spatiotemporal matrix, and then constructing a weight matrix; using the weight matrix to perform weighted moving average filtering on effective pixel grayscale values of each image frame; and using an optical flow method to perform displacement calculation on the filtered effective pixel grayscale values of each image frame to obtain a pixel displacement, and then performing conversion by means of the scale parameter to obtain a physical displacement.

Description

A non-contact dynamic measurement method, system, processing equipment, and storage medium for structural deformation based on ultrasensitive optical flow. Technical Field

[0001] This invention relates to a non-contact dynamic measurement method, system, processing device, and storage medium for structural deformation based on ultrasensitive optical flow. Background Technology

[0002] Displacement is a key parameter in engineering structural design and performance evaluation, directly reflecting the basic performance and state of a structure. Existing displacement measurement technologies are mainly divided into two categories: contact and non-contact. Contact displacement measurement technologies include displacement gauges, inclinometers, and levels, but their practical application is limited due to numerous problems. For example, displacement gauges require fixed supports, making practical application difficult. Non-contact measurement technologies include GPS and lidar ranging, but they also have many shortcomings. For instance, GPS has low accuracy in measuring deflection, and its measurement results are unstable due to many external factors, such as satellite coverage, weather conditions, and multiple reflections; lidar requires specific measurement angles and specialized personnel to operate it with corner reflectors; and lidar ranging is limited by the short penetration distance of laser light.

[0003] Vision-based image measurement methods are widely used in displacement measurement due to their low cost and ease of implementation. Existing methods mainly include digital image correlation (DIC) and optical flow (OF). The DIC method typically requires a target with a random speckle pattern to be placed on the surface of the object being measured, and it is widely used in structural deformation measurement. However, due to this dependence on the target, the number of spatial measurement points in the DIC method is severely limited by the number of targets. Furthermore, because the algorithm involves a large amount of correlation calculation, the computational efficiency of the DIC method is relatively low, making it difficult to meet the requirements of high-sampling-rate online measurements.

[0004] Summary of the Invention

[0005] The purpose of this invention is to provide a non-contact dynamic measurement method for structural deformation based on ultrasensitive optical flow, thereby solving the problems mentioned in the background art. To achieve the above objective, this invention provides the following technical solution:

[0006] A non-contact dynamic measurement method for structural deformation based on ultrasensitive optical flow includes the following steps:

[0007] Acquire images of the structure to be tested;

[0008] The scale parameter p is calibrated based on the physical dimensions of the structure under test and the number of pixels occupied by the image;

[0009] Based on the acquired images, N is continuously extracted. t Frame image; for each frame image, the spatial gradient values ​​of all pixels within the selected region are calculated. These spatial gradient values ​​contain two vertical components g. x and g y Set the grayscale gradient threshold g c Spatial gradient values ​​greater than the threshold g c The pixels selected are the valid pixels, and the number of valid pixels is represented as N. s Based on N t The effective pixel grayscale values ​​of the frame image are used to construct an effective pixel grayscale spatiotemporal matrix M, which has a size of N. t ×N s ;

[0010] Singular value decomposition is performed based on the effective pixel gray-level spatiotemporal matrix M to obtain the left singular matrix U, the singular value diagonal matrix Σ, and the right singular matrix V; based on all elements of the singular value diagonal matrix Σ... Obtain the total energy of all singular values. Set the threshold value of the signal mode singular energy and its proportion of the total singular energy to R. E According to the descending order of singular values, the energy and proportion of the first r order singular values ​​can be satisfied. Greater than the proportional threshold R E The minimum value of r is chosen as the number of signal modes; the matrix V is composed of the first r column vectors of the right singular matrix V. r Constructing a weight matrix based on deformation recognition

[0011] A weighted average filter is performed based on the effective pixel grayscale values ​​of each frame of the image to obtain the filtered effective pixel grayscale value I. f =IW; The pixel displacement time history is obtained by calculating the displacement of the effective pixel grayscale values ​​after filtering in each frame of the image based on the optical flow method. The calculation formula is as follows: Based on the scale parameter p and pixel displacement s, the physical displacement time history d is calculated, where the calculation formula is: d = s * p.

[0012] Furthermore, the specific steps for acquiring images of the structure under test are as follows: a camera is set up on the plane where the structure under test is located, the camera's viewing angle and focal length are adjusted so that the camera's imaging plane overlaps with the plane of the structure under test, and the camera is turned on to continuously acquire images of the structure under test.

[0013] The physical dimension of the structure under test is the horizontal width w of the beam structure, and the number of pixels occupied by the image is N. w According to the formula p = w / N w Calculate the scale parameter p, and measure the horizontal width w and the corresponding number of pixels N multiple times along the axial direction of the beam structure. wThe average value is used as the final scale parameter value.

[0014] Furthermore, the R E It is 95%.

[0015] This invention also provides a non-contact dynamic measurement system for structural deformation based on ultrasensitive optical flow, comprising:

[0016] The image acquisition module is used to acquire images of the structure under test.

[0017] The scale parameter calibration module is used to calibrate the scale parameter p based on the physical size of the structure under test and the number of pixels occupied by the image;

[0018] The effective pixel grayscale spatiotemporal matrix construction module is used to continuously extract N pixels from the acquired image. t Frame image; for each frame image, the spatial gradient values ​​of all pixels within the selected region are calculated. These spatial gradient values ​​contain two vertical components g. x and g y Set the grayscale gradient threshold g c Spatial gradient values ​​greater than the threshold g c The pixels selected are the valid pixels, and the number of valid pixels is represented as N. s Based on N t The effective pixel grayscale values ​​of the frame image are used to construct an effective pixel grayscale spatiotemporal matrix M, which has a size of N. t ×N s ;

[0019] The weight matrix construction module is used to perform singular value decomposition based on the effective pixel gray-level spatiotemporal matrix M, obtaining the left singular matrix U, the singular value diagonal matrix Σ, and the right singular matrix V; based on all elements of the singular value diagonal matrix Σ... Obtain the total energy of all singular values. Set the threshold value of the signal mode singular energy and its proportion of the total singular energy to R. E According to the descending order of singular values, the energy and proportion of the first r order singular values ​​can be satisfied. Greater than the proportional threshold R E The minimum value of r is chosen as the number of signal modes; the matrix V is composed of the first r column vectors of the right singular matrix V. r Constructing a weight matrix based on deformation recognition

[0020] The physical displacement time history calculation module is used to perform weighted average filtering calculation based on the effective pixel grayscale values ​​of each frame of the image to obtain the filtered effective pixel grayscale value I. f =IW; The pixel displacement time history is obtained by calculating the displacement of the effective pixel grayscale values ​​after filtering in each frame of the image based on the optical flow method. The calculation formula is as follows: Based on the scale parameter p and pixel displacement s, the physical displacement time history d is calculated, where the calculation formula is: d = s * p.

[0021] The present invention also provides a processing device, comprising: a memory and a processor, wherein the memory stores a computer program executable by the processor, and the processor executes the computer program to implement the non-contact dynamic measurement method for structural deformation based on the ultrasensitive optical flow method described above.

[0022] The present invention also provides a storage medium storing a computer program, which is executed by a controller to perform the above-described non-contact dynamic measurement method for structural deformation based on ultrasensitive optical flow. Beneficial effects:

[0023] This invention overcomes the shortcomings of traditional contact-based structural deformation measurement, such as limited spatial measurement points, cumbersome equipment layout, and complex acquisition systems. Compared with existing vision-based image measurement methods, it does not require the placement of patterned targets on the structure but directly utilizes the surface texture or edge of the structure to achieve truly non-contact, full-field dense (high-resolution) measurement.

[0024] The ultrasensitive optical flow method proposed in this invention breaks through the sensitivity limit of image measurement methods, significantly improving measurement accuracy. Building upon the advantages of targetless measurement and high spatial resolution, it ensures measurement accuracy. This is particularly important for long-distance measurement of large structures, where the high field of view required for full-field measurement of large structures, coupled with the extremely small pixel displacement corresponding to structural deformation, necessitates very high measurement accuracy.

[0025] The calculation process of the method proposed in this invention does not involve repeated correlation calculations, but only linear transformation and multi-pixel parallel calculation, which has high computational efficiency and can be used for high-frequency real-time measurement of structural deformation. Attached Figure Description

[0026] Figure 1 is a structural diagram of the test equipment of the present invention;

[0027] Figure 2 is a frame selection diagram of the first frame of the present invention;

[0028] Figure 3 is an example diagram of the construction of the effective pixel grayscale spatiotemporal matrix model of the present invention;

[0029] Figure 4 is an example diagram of constructing the effective pixel weight matrix model of the present invention;

[0030] Figure 5 is an example diagram of the weighted average filtering model of the present invention;

[0031] Figure 6 is an example of the test results of the present invention. Detailed Implementation

[0032] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] This invention provides a non-contact dynamic measurement method for structural deformation based on ultrasensitive optical flow, comprising the following steps:

[0034] S1: Acquire an image of the structure to be tested;

[0035] Specifically, a camera is set up on the plane of the structure to be tested, and the camera's angle and focal length are adjusted so that the camera's imaging plane overlaps with the plane of the structure to be tested, and the camera is turned on to continuously acquire images of the structure to be tested.

[0036] In one example: the measurement object is a beam structure fixed at both ends, and the excitation method is a modal exciter that horizontally excites the mid-span of the beam structure. The excitation signal is a simple harmonic signal (frequency 5Hz). To better illustrate and demonstrate the method of the present invention, a simple black and white striped sticker is affixed to the upper surface of the beam structure (not necessary in actual measurement). The horizontal deformation vibration of the beam structure under horizontal excitation is measured. A video camera is fixed on the top of the beam structure with a tripod, and the lens is perpendicular to the horizontal plane. The camera model is SONY PXW-FS5M2, with a CMOS imaging chip resolution of 1920×1080 pixels, a lens focal length of 24 mm, a frame rate of 120 frames / second, and an image signal bit depth of 8 bits. To verify the accuracy of the measurement results of the method of the present invention, a laser Doppler vibration meter (a high-precision interferometric optical single-point measurement system, whose price is nearly 100 times that of the camera measurement system) is arranged in front of the beam structure. At the same time, the horizontal vibration displacement of the mid-span of the beam structure is measured, as shown in Figure 1.

[0037] S2: Based on the physical dimensions of the structure under test and the number of pixels occupied by the image, calibrate the scale parameter p (unit: mm / pixel).

[0038] In one example: Without loading, using the known horizontal width w of the beam structure and the number of pixels N it occupies in the image. w According to the formula p = w / N w Determine the dimensional parameters, and measure the horizontal width w and the corresponding number of pixels N multiple times along the axial direction of the beam structure. w The average value is used as the final scale parameter value.

[0039] S3: Continuous extraction of N based on acquired images t Frame image; for each frame image, the spatial gradient values ​​of all pixels within the selected region are calculated. These spatial gradient values ​​contain two vertical components g.x and g y Set the grayscale gradient threshold g c Spatial gradient values ​​greater than the threshold g c The pixels selected are the valid pixels, and the number of valid pixels is represented as N. s Based on N t The effective pixel grayscale values ​​of the frame image are used to construct an effective pixel grayscale spatiotemporal matrix M, which has a size of N. t ×N s ;

[0040] S4: Perform singular value decomposition based on the effective pixel grayscale spatiotemporal matrix M to obtain the left singular matrix U, the singular value diagonal matrix ∑, and the right singular matrix V; based on all elements of the singular value diagonal matrix ∑... Obtain the total energy of all singular values. Set the threshold value of the signal mode singular energy and its proportion of the total singular energy to R. E According to the descending order of singular values, the energy and proportion of the first r order singular values ​​can be satisfied. Greater than the proportional threshold R E The minimum value of r is chosen as the number of signal modes; the matrix V is composed of the first r column vectors of the right singular matrix V. r Constructing a weight matrix based on deformation recognition

[0041] In one example: Multiple consecutively acquired frames of images are imported into the computer. A calculation region is selected within the first frame (as shown in Figure 2). The grayscale gradient value of each pixel within this region is calculated. Pixels with a grayscale gradient greater than a specified threshold are selected as valid pixels (in this embodiment, pixels at the intersection of black and white stripes). The grayscale values ​​of the valid pixels in each frame are extracted to form the original grayscale spatiotemporal matrix M of the valid pixels, as shown in Figure 3. Singular value decomposition is performed on this spatiotemporal matrix. Based on the condition that the sum of singular value energies is not less than 95%, the minimum number of singular modes that meet this condition are selected as signal modes, forming the mode set V. r And construct the weight matrix See Figure 4 for details.

[0042] S5: Perform a weighted average filtering calculation based on the effective pixel grayscale values ​​of each frame image to obtain the filtered effective pixel grayscale value I. f =IW; The displacement time history (unit: pixels) is obtained by calculating the displacement of the effective pixel grayscale values ​​after filtering in each frame of the image based on the optical flow method. The calculation formula is as follows: Where s represents displacement, x j ,y k Represents the position coordinates of the pixel in the j-th column and k-th row, and t represents time. Indicates the grayscale value of the selected initial frame image after filtering, If Represents the grayscale of the calculated frame image after filtering. The initial frame grayscale gradient after filtering is represented by ; the physical displacement time history d (unit: millimeters) is calculated based on the scale parameter p and pixel displacement s, where the calculation formula is: d = s * p

[0043] In one example: the weight matrix is ​​used to perform a weighted average filter M on the original grayscale values. f =MW, and then replace its gray value in the original image with the filtered gray value to obtain the filtered multi-frame image, as shown in Figure 5; the effective pixel displacement of each frame image is calculated using the gradient-based optical flow method, and finally converted to physical length units through the scale parameter. Figure 6 shows the measurement results of the method of the present invention converted to physical length units, and the measurement results of the laser Doppler vibrometer used for comparison. It can be found that under the large amplitude vibration condition of 0.2mm, the measurement results of the method of the present invention are almost completely consistent with the results of the laser Doppler vibrometer; under the small amplitude condition of 0.016mm, the measurement results of the method of the present invention are generally consistent with the results of the laser Doppler vibrometer, with the displacement at the peak being slightly smaller than that of the laser Doppler vibrometer.

[0044] This invention also provides a non-contact dynamic measurement system for structural deformation based on ultrasensitive optical flow method, comprising:

[0045] The image acquisition module is used to acquire images of the structure under test.

[0046] The scale parameter calibration module is used to calibrate the scale parameter p based on the physical size of the structure under test and the number of pixels occupied by the image;

[0047] The effective pixel grayscale spatiotemporal matrix construction module is used to continuously extract N pixels from the acquired image. t Frame image; for each frame image, the spatial gradient values ​​of all pixels within the selected region are calculated. These spatial gradient values ​​contain two vertical components g. x and g y Set the grayscale gradient threshold g c Spatial gradient values ​​greater than the threshold g c The pixels selected are the valid pixels, and the number of valid pixels is represented as N. s Based on N t The effective pixel grayscale values ​​of the frame image are used to construct an effective pixel grayscale spatiotemporal matrix M, which has a size of N. t ×N s ;

[0048] The weight matrix construction module is used to perform singular value decomposition based on the effective pixel gray-level spatiotemporal matrix M, obtaining the left singular matrix U, the singular value diagonal matrix Σ, and the right singular matrix V; based on all elements of the singular value diagonal matrix Σ... Obtain the total energy of all singular values. Set the threshold value of the signal mode singular energy and its proportion of the total singular energy to R. E According to the descending order of singular values, the energy and proportion of the first r order singular values ​​can be satisfied. Greater than the proportional threshold R E The minimum value of r is chosen as the number of signal modes; the matrix V is composed of the first r column vectors of the right singular matrix V. r Constructing a weight matrix based on deformation recognition

[0049] The physical displacement time history calculation module is used to perform weighted average filtering calculation based on the effective pixel grayscale values ​​of each frame of the image to obtain the filtered effective pixel grayscale value I. f =IW; The pixel displacement time history is obtained by calculating the displacement of the effective pixel grayscale values ​​after filtering in each frame of the image based on the optical flow method. The calculation formula is as follows: Based on the scale parameter p and pixel displacement s, the physical displacement time history d is calculated, where the calculation formula is: d = s * p.

[0050] The present invention also provides a processing device, comprising: a memory and a processor, wherein the memory stores a computer program executable by the processor, and the processor executes the computer program to implement the non-contact dynamic measurement method for structural deformation based on the ultrasensitive optical flow method described above.

[0051] The present invention also provides a storage medium storing a computer program, which is executed by a controller to perform the above-described non-contact dynamic measurement method for structural deformation based on ultrasensitive optical flow.

[0052] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. The scope of protection of the present invention should be determined by the scope of the claims. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

A non-contact dynamic measurement method for structural deformation based on ultrasensitive optical flow method, characterized in that, Includes the following steps: Acquire images of the structure to be tested; The scale parameter p is calibrated based on the physical dimensions of the structure under test and the number of pixels occupied by the image; Based on the continuously extracted N images t Frame image; for each frame image, the spatial gradient values ​​of all pixels within the selected region are calculated. These spatial gradient values ​​contain two vertical components g. x and g y Set the grayscale gradient threshold g c Spatial gradient values ​​greater than the threshold g c The pixels selected are the valid pixels, and the number of valid pixels is represented as N. s Based on N t The effective pixel grayscale values ​​of the frame image are used to construct an effective pixel grayscale spatiotemporal matrix M, the size of which is N. t ×N s ; Singular value decomposition is performed based on the effective pixel gray-level spatiotemporal matrix M to obtain the left singular matrix U, the singular value diagonal matrix ∑, and the right singular matrix V; based on all elements of the singular value diagonal matrix ∑... Obtain the total energy of all singular values. Set the threshold value of the signal mode singular energy and its proportion of the total singular energy to R. E According to the descending order of singular values, the energy and proportion of the first r order singular values ​​can be satisfied. Greater than the proportional threshold R E The minimum value of r is chosen as the number of signal modes; the matrix V is composed of the first r column vectors of the right singular matrix V. r Constructing a weight matrix based on deformation recognition A weighted average filter is performed based on the effective pixel grayscale values ​​of each frame of the image to obtain the filtered effective pixel grayscale value I. f =IW; The pixel displacement time history is obtained by calculating the displacement of the effective pixel grayscale values ​​after filtering in each frame of the image based on the optical flow method. The calculation formula is as follows: Based on the scale parameter p and pixel displacement s, the physical displacement time history d is calculated, where the calculation formula is: d = s * p. The non-contact dynamic measurement method for structural deformation based on ultrasensitive optical flow method according to claim 1 is characterized in that, The specific steps for acquiring images of the structure under test are as follows: set up a camera on the plane where the structure under test is located, adjust the camera's viewing angle and focal length so that the camera's imaging plane overlaps with the plane of the structure under test, and turn on the camera to continuously acquire images of the structure under test. The non-contact dynamic measurement method for structural deformation based on ultrasensitive optical flow method according to claim 1 is characterized in that, The physical dimension of the structure under test is the horizontal width w of the beam structure, and the number of pixels occupied by the image is N. w According to the formula p = w / N w Calculate the scale parameter p, and measure the horizontal width w and the corresponding number of pixels N multiple times along the axial direction of the beam structure. w The average value is used as the final scale parameter value. The non-contact dynamic measurement method for structural deformation based on ultrasensitive optical flow method according to claim 1 is characterized in that, The R E It is 95%. A non-contact dynamic measurement system for structural deformation based on ultrasensitive optical flow method, characterized in that... include The image acquisition module is used to acquire images of the structure under test. The scale parameter calibration module is used to calibrate the scale parameter p based on the physical size of the structure under test and the number of pixels occupied by the image; The effective pixel grayscale spatiotemporal matrix construction module is used to continuously extract N pixels from the acquired image. t Frame image; for each frame image, the spatial gradient values ​​of all pixels within the selected region are calculated. These spatial gradient values ​​contain two vertical components g. x and g y Set the grayscale gradient threshold g c Spatial gradient values ​​greater than the threshold g c The pixels selected are the valid pixels, and the number of valid pixels is represented as N. s Based on N t The effective pixel grayscale values ​​of the frame image are used to construct an effective pixel grayscale spatiotemporal matrix M, the size of which is N. t ×N s ; The weight matrix construction module is used to perform singular value decomposition based on the effective pixel gray-level spatiotemporal matrix M, obtaining the left singular matrix U, the singular value diagonal matrix ∑, and the right singular matrix V; based on all elements of the singular value diagonal matrix ∑... Obtain the total energy of all singular values. Set the threshold value of the signal mode singular energy and its proportion of the total singular energy to R. E According to the descending order of singular values, the energy and proportion of the first r order singular values ​​can be satisfied. Greater than the proportional threshold R E The minimum value of r is chosen as the number of signal modes; the matrix V is composed of the first r column vectors of the right singular matrix V. r Constructing a weight matrix based on deformation recognition The physical displacement time history calculation module is used to perform weighted average filtering calculation based on the effective pixel grayscale values ​​of each frame of the image to obtain the filtered effective pixel grayscale value I. f =IW; The pixel displacement time history is obtained by calculating the displacement of the effective pixel grayscale values ​​after filtering in each frame of the image based on the optical flow method. The calculation formula is as follows: Based on the scale parameter p and pixel displacement s, the physical displacement time history d is calculated, where the calculation formula is: d = s * p. Processing equipment, including: The system includes a memory and a processor, wherein the memory stores a computer program executable by the processor, and the processor executes the computer program to implement the non-contact dynamic measurement method for structural deformation based on the ultrasensitive optical flow method as described in claim 1. A storage medium storing a computer program, which is executed by the controller as described in claim 1, for non-contact dynamic measurement of structural deformation based on ultrasensitive optical flow.

Images