Large-scale facility dynamic vision displacement measurement method based on unmanned aerial vehicle

Abstract

The invention provides a large-scale facility dynamic vision displacement measurement method based on an unmanned aerial vehicle, and relates to the field of unmanned aerial vehicle displacement measurement, and the method comprises the steps: collecting unmanned aerial vehicle images before and after the displacement of a to-be-measured large-scale facility; performing motion compensation on the unmanned aerial vehicle image; and based on a sampling Moire method and a maximum cross-correlation method, the unmanned aerial vehicle image after motion compensation is processed to obtain displacement data of the to-be-measured large-scale facility, and displacement measurement of the to-be-measured large-scale facility is completed. The sampling Moire method is adopted to carry out sub-pixel-level displacement measurement, so that the measurement precision is improved, and aiming at the problem that the sampling Moire method is influenced by light and equipment in practical application and the displacement calculation is inaccurate due to the fact that a moire fringe image is not ideal, the image matching method is utilized to carry out phase calculation, so that the robustness of the method is enhanced.

Description

Technical Field

[0001] This application relates to the field of UAV displacement measurement, and more particularly to a method for dynamic visual displacement measurement of large facilities based on UAVs. Background Technology

[0002] Large-scale facilities (such as bridges and buildings) may deform or displace under external loads. Timely and accurate measurement of these displacements is crucial for the structural safety assessment and maintenance of such facilities. Structural health monitoring involves deploying various types of sensors on the structure to collect real-time data on its mechanical responses, including displacement, vibration acceleration, strain, and modal parameters. This data is used for structural damage diagnosis and condition assessment to determine whether the load-bearing capacity meets normal functional requirements. Displacement is a key indicator for structural condition assessment and performance evaluation. The static and dynamic characteristics of a structure, such as load-bearing capacity, deflection, deformation, load distribution, load input, influence lines, influence surfaces, and modal parameters, can be calculated using structural displacements and further converted into corresponding physical indicators for structural safety assessment.

[0003] Non-contact displacement measurement methods mainly include Global Positioning System (GPS), laser ranging, three-dimensional laser scanning, microwave radar, permanent scatterer radar interferometry (PS-InSAR) technology, computer vision methods, etc.

[0004] With the continuous development of computer vision technology and image acquisition equipment, computer vision-based structural displacement monitoring methods are emerging and have been validated in practical engineering applications. The principle of computer vision-based displacement measurement methods is to perform target tracking processing on video of the structure under test captured by a camera to obtain the motion trajectory of the measurement point in the image, and then determine the displacement information of the structure through the geometric relationship between the image and the real world.

[0005] To address the limitations of fixed-camera-based non-contact measurement methods in structural health monitoring of large facilities (such as long-span bridges), a non-contact visual measurement system based on an unmanned aerial vehicle (UAV) platform has been proposed. Existing UAV platforms offer advantages such as low cost and high mobility, and can reach targets inaccessible by manual measurement, effectively solving the problem of the difficulty in installing fixed cameras on large structures.

[0006] Compared to traditional fixed reference point measurement methods, UAV systems operate from non-fixed observation points. The movement of the UAV itself degrades the image quality and target detection and localization accuracy. Therefore, the key to transforming visual displacement measurement from a static to a non-static measurement point lies in analyzing the impact of motion on target detection and localization, and compensating for and correcting motion errors in the displacement measurement results to obtain accurate dynamic displacement information of the structure. Summary of the Invention

[0007] The purpose of this invention is to provide a dynamic visual displacement measurement method for large facilities based on UAVs, in order to solve the problem of low accuracy in existing UAV-based methods for measuring displacement differences of large facilities.

[0008] The above-mentioned objective of this application is achieved through the following technical solution: S1: Collect UAV images of the large facility under test before and after displacement; S2: Perform motion compensation on drone images; S3: Based on the sampling moiré method and the maximum cross-correlation method, the motion-compensated UAV imagery is processed to obtain the initial displacement data; S4: Based on the sampling moiré method and the maximum cross-correlation method, combined with the initial displacement data, subpixel-level motion compensation is performed on the UAV imagery to complete the displacement measurement of the large facility under test.

[0009] This application uses the above technical solution, which is divided into two steps through two compensation steps, including: using a similarity transformation method to complete pixel-level compensation; and combining the sampling moiré method to calculate the sub-pixel displacement of the reference mark at the fixed position, which is the sub-pixel-level image displacement caused by the movement of the UAV. The second step of compensation is completed by subtracting the displacement of the reference mark at the fixed position from the displacement of the mark at the test point.

[0010] Optionally, step S1 includes: S11: Affix a set of markers to the large facility to be tested; A fixed position reference mark is selected at a location in the large facility under test where no structural displacement occurs. There are two fixed position reference markers; Mark the points to be tested in the large facility to be tested; S12: Take drone images of the large facility under test before and after deformation using a drone; The UAV imagery includes: fixed position reference markers and markers for the points to be measured.

[0011] Optionally, step S2 includes: S21: Locate the fixed position reference marker in the UAV imagery, and determine the center coordinates and reference line; S22: Use the initial frame image in the UAV imagery as the reference image, and the other images as the target images; S23: Simplify the six degrees of freedom motion of the UAV to four degrees of freedom, which includes: motion along the x-axis. Motion along the y-axis Rotation angle and image scaling The four degrees of freedom are mapped to the translation, rotation, and scaling transformations of UAV imagery. S24: Align the target image with the reference image using a similarity transformation algorithm, combined with the center coordinates and reference lines.

[0012] Optionally, step S3 includes: S31: Extract keyframe images from UAV imagery; S32: Scale up or down the keyframe image proportionally to generate moiré fringes; By downsampling and intensity interpolation, multiple phase-shifted moiré fringes of the keyframe image are obtained to obtain the moiré fringe image; S33: The template matching method is used to calculate the phase of the moiré fringes in the moiré fringe image to obtain the phase difference of the moiré fringes; S34: By using the phase difference of the moiré fringes and combining it with the principle of sampling moiré, the phase difference of the moiré fringes is converted into the relationship between the displacement, and the initial displacement data is obtained.

[0013] Optionally, step S33 includes: A complete cycle of moiré fringe image is constructed using the moiré fringe image as a template image; The moiré fringe images before and after displacement are matched with the template image respectively; By combining the maximum cross-correlation method, the center point coordinates of the moiré fringe image are located by indexing the peak values ​​of the correlation coefficients. The phase difference of the moiré fringes is obtained by calculating the shift of the center point coordinates.

[0014] This application addresses the problem of inaccurate displacement calculation caused by the non-ideal moiré fringe image due to the influence of light and equipment in practical applications of the sampling moiré method. By adopting the above technical solution, the application utilizes image matching and maximum cross-correlation methods for phase calculation, thereby enhancing the robustness of the method.

[0015] Optionally, step S34 includes:

[0016] in This refers to the initial displacement data, i.e., the grating motion data; Indicates the grating period; This represents the phase difference of the moiré fringes.

[0017] Optionally, step S4 includes: S41: Extract a moiré fringe image with fixed position reference marks from the moiré fringe image obtained in step S32; S42: The template matching method calculates the phase of the moiré fringes in a moiré fringe image with fixed position reference marks to obtain the phase difference of the moiré fringes; S43: By using the phase difference of the moiré fringes and combining it with the sampling moiré method, the phase difference of the moiré fringes is converted into the relationship between the displacement and the displacement, resulting in sub-pixel-level image displacement caused by the drone's motion. S44: Subtract the initial displacement data from the subpixel-level image displacement to obtain the displacement data of the large facility under test, thus completing the subpixel-level displacement compensation.

[0018] An electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to enable the electronic device to perform a dynamic visual displacement measurement method for large facilities based on unmanned aerial vehicles (UAVs).

[0019] A computer-readable storage medium storing instructions that, when executed, perform a method for dynamic visual displacement measurement of large facilities based on unmanned aerial vehicles (UAVs).

[0020] The beneficial effects of the technical solution provided in this application are: Displacement monitoring of large-scale facilities structures is achieved using a dynamic visual measurement method based on unmanned aerial vehicles (UAVs). To address errors caused by UAV motion during dynamic measurement, a registration method is used on the acquired video images, combined with sampled moiré pattern (SMB) for UAV motion compensation. Compared to inertial navigation and GNSS navigation methods, this approach is easier to apply to images and can achieve sub-pixel accuracy. Furthermore, the SMB method for sub-pixel displacement measurement offers advantages over intensity-based visual measurement methods such as optical flow and DIC methods, including higher accuracy, robustness, lower cost, and faster speed. In addition, while retaining the flexibility of UAV visual measurement methods, this approach improves the measurement errors caused by UAV motion, simplifies the measurement process, and enhances both measurement efficiency and accuracy. Attached Figure Description

[0021] The present application will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a schematic diagram of the sampling moiré method in the embodiments of this application; Figure 2 These are raster images before and after displacement in the embodiments of this application; Figure 3 These are moiré fringe patterns before and after displacement in the embodiments of this application; Figure 4 This is a phase comparison diagram of the moiré fringes before and after displacement in the embodiments of this application; Figure 5 This is a group of fixed marks on the object under test in the embodiments of this application; Figure 6This is a reference line diagram in the embodiments of this application; Figure 7 This is a motion compensation diagram from an embodiment of this application; Figure 8 This is a template matching method diagram in the embodiments of this application; Figure 9 This is the sampling moiré plot in the embodiments of this application; Figure 10 This is the ideal moiré fringe grayscale curve diagram in the embodiments of this application; Figure 11 This is the actual moiré fringe grayscale curve diagram in the embodiments of this application; Figure 12 This is the center extraction map of the maximum cross-correlation method in the embodiments of this application; Figure 13 This is a diagram illustrating the fringe positioning effect of the three-dimensional peak-finding method in an embodiment of this application. Figure 14 This is an image of an ideal calibration board in the embodiments of this application; Figure 15 This is the moiré fringe pattern in the embodiments of this application; Figure 16 This is a schematic diagram of the electronic device structure in the embodiments of this application. Detailed Implementation

[0022] To provide a clearer understanding of the technical features, objectives, and effects of this application, the specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0023] The embodiments of this application provide a method for dynamic visual displacement measurement of large facilities based on unmanned aerial vehicles (UAVs).

[0024] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating the steps of a method for measuring the dynamic visual displacement of a large facility based on an unmanned aerial vehicle (UAV) according to an embodiment of this application, including: S1: Collect UAV images of the large facility under test before and after displacement; S2: Perform motion compensation on drone images; S3: Based on the sampling moiré method and the maximum cross-correlation method, the motion-compensated UAV imagery is processed to obtain the initial displacement data; S4: Based on the sampling moiré method and the maximum cross-correlation method, combined with the initial displacement data, subpixel-level motion compensation is performed on the UAV imagery to complete the displacement measurement of the large facility under test.

[0025] As one embodiment, to improve displacement measurement accuracy, sub-pixel-level precision marker center coordinates are extracted using the sampling moiré method, and the coordinate trajectory of each marker is extracted using the maximum normalized cross-correlation algorithm. These trajectories include the motion of the UAV camera and the structural bridge displacement caused by the load. The UAV motion data is separated to obtain UAV motion compensation results for accurate displacement measurement.

[0026] Step S1 includes: S11: Affix a set of markers to the large facility to be tested; A fixed position reference mark is selected at a location in the large facility under test where no structural displacement occurs. There are two fixed position reference markers; Mark the points to be tested in the large facility to be tested; As one embodiment, a set of markers is fixed on the object to be tested, such as... Figure 5 As shown, reference marks 1 and 2 are fixed at positions on the structure under test that do not undergo structural displacement or can serve as the origin of a reference system. Markers 3 and 4 are fixed at the points to be measured. If there are multiple points to be measured in the same plane, multiple marks can be fixed to record the structural displacement caused by external loads and the motion of the UAV camera. Therefore, the position of the reference line formed by connecting the centers of fixed marks 1 and 2 remains unchanged before and after the structural displacement of the structure under test occurs.

[0027] S12: Take drone images of the large facility under test before and after deformation using a drone; The UAV imagery includes: fixed position reference markers and markers for the points to be measured.

[0028] In one embodiment of this application, a drone is used to capture video, recording images before and after deformation. The resulting images record the motion information of the object under test and the drone. The geometry of the object under test and the hovering camera settings are characterized. By marking the planar structure of the supporting bridge side and constraining the hovering mode used for aerial photography, the six degrees of freedom motion of the drone is simplified to four degrees of freedom.

[0029] Step S2 includes: S21: Locate the fixed position reference marker in the UAV imagery, and determine the center coordinates and reference line; In one embodiment of this application, image processing is performed frame by frame on the acquired video. First, reference markers are located and their center coordinates are extracted. Simultaneously, an initial frame is determined as a reference image for image alignment. Reference lines are formed by connecting the centers of the fixed markers, such as... Figure 6 As shown, the position of the reference line formed by connecting the centers of fixed marks 1 and 2 remains unchanged before and after the structure under test undergoes structural displacement.

[0030] S22: Use the initial frame image in the UAV imagery as the reference image, and the other images as the target images; S23: Simplify the six degrees of freedom motion of the UAV to four degrees of freedom, which includes: motion along the x-axis. Motion along the y-axis Rotation angle and image scaling The four degrees of freedom are mapped to the translation, rotation, and scaling transformations of UAV imagery. In one embodiment of this application, the initial frame image in the captured video serves as a reference image, recording the positions of the structure under test and the reference marker before displacement occurs. During filming, the four-degree-of-freedom motion of the drone itself causes the image to translate, rotate, or scale. Taking image rotation as an example... Figure 7 As shown, after deformation occurs, reference markers 3 and 4 (markers of the points to be measured) move and shift from the reference line. The displacement of markers 3 and 4 is the displacement of the structure to be measured. At the same time, due to the movement of the UAV, the image rotates relative to the initial frame.

[0031] S24: Align the target image with the reference image using a similarity transformation algorithm, combined with the center coordinates and reference lines.

[0032] In one embodiment of this application, since the drone motion is simplified to four degrees of freedom and corresponds to the translation, rotation, and scaling transformations of the image, the displacement caused by the drone motion can be compensated through a similarity transformation algorithm. By aligning the reference lines before and after deformation through similarity transformation, all frames of the captured video are aligned with the initial frame, achieving pixel-level precision displacement compensation.

[0033] Step S3 includes: S31: Extract keyframe images from UAV imagery; As one example, in actual measurement, a checkerboard calibration board is used as a regular pattern, i.e., a reference mark used for measurement. Video is captured with the reference mark as the shooting target, and key frame images are extracted for displacement analysis.

[0034] As one embodiment, the sampling moiré method measurement steps are as follows: Figure 9 As shown. First, the target calibration board image is low-pass filtered to remove noise. A suitable sampling interval is selected based on the size of the calibration board image, aiming to satisfy the mathematical relationship that the grating spacing is 4.5 times the sampling interval.

[0035] S32: Scale up or down the keyframe image proportionally to generate moiré fringes; In one embodiment of this application, the moiré fringe period is increased by magnifying or reducing the acquired image and adjusting the ratio between the grating spacing and the number of sampling rows. When applying the sampled moiré method for displacement measurement, the displacement to be measured is expressed as:

[0036] Where u is the displacement to be measured, and p is the moiré fringe period. Therefore, increasing the moiré fringe period can improve the measurement accuracy of the sampling moiré method.

[0037] According to the principle of sampled moiré measurement, the moiré fringes obtained by downsampling difference have a definite mathematical relationship with the grating period when the sampling interval is fixed. Studies have shown that, within a certain range, the larger the grating period, the smaller the moiré fringe period obtained by the sampling difference.

[0038] Taking a sampling interval k=4 (i.e., taking 1 row / column every 4 rows / columns) as an example, the mathematical relationship between the grating spacing and the number of sampling rows / columns is as follows: when the grating spacing is 5 pixels, the moiré fringe period is 20 pixels; when the grating spacing is 4.5 pixels, the moiré fringe period is 36 pixels; when the grating spacing is 4.3 pixels, the moiré fringe period is 60 pixels; and when the grating spacing is 4.2 pixels, the moiré fringe period is 84 pixels. Therefore, the closer the ratio of the grating spacing to the number of sampling rows / columns is to 4, the larger the moiré fringe period. By appropriately selecting the magnification / reduction factor of the acquired image, reducing the ratio of the grating spacing to the number of sampling rows / columns can improve the measurement accuracy of the sampled moiré method.

[0039] By downsampling and intensity interpolation, multiple phase-shifted moiré fringes of the keyframe image are obtained to obtain the moiré fringe image; As one embodiment, the image is scaled up or down proportionally to ensure that moiré fringes can be generated. Multiple phase-shifted moiré fringes are obtained using downsampling and intensity interpolation techniques.

[0040] S33: The template matching method is used to calculate the phase of the moiré fringes in the moiré fringe image and obtain the phase difference of the moiré fringes.

[0041] S34: By using the phase difference of the moiré fringes and combining it with the principle of sampling moiré, the phase difference of the moiré fringes is converted into the relationship between the displacement, and the initial displacement data is obtained.

[0042] Step S33 includes: As an example, in actual shooting and image processing, due to the influence of experimental equipment and lighting, the grayscale values ​​of moiré fringes obtained from image processing usually differ significantly from the sinusoidal image, resulting in a large error in the image representation in the phase calculation formula (1), which affects the phase calculation results. To address this problem, this invention proposes a template matching method for phase calculation, the principle of which is as follows: Figure 8 As shown.

[0043] A complete cycle of moiré fringe image is constructed using the moiré fringe image as a template image; The moiré fringe images before and after displacement are matched with the template image respectively; By combining the maximum cross-correlation method, the center point coordinates of the moiré fringe image are located by indexing the peak values ​​of the correlation coefficients. The phase difference of the moiré fringes is obtained by calculating the movement of the center point coordinates. As one embodiment, if multiple test points need to be measured, multiple moiré fringe images need to be acquired simultaneously when calculating displacement using the template matching method. Since the criteria for identifying moiré fringes are complex, the moiré fringes can be located by examining the calibration plate image before the sampling difference. Because the maximum cross-correlation method can only identify the coordinates of a single matched image, for the extraction of multiple sites, this invention performs three-dimensional peak finding processing on the correlation coefficient data between the captured image and the calibration plate image to obtain the center coordinates of all calibration plates. Locating the calibration plate position allows for the location of the moiré fringes. The cropping area of ​​each moiré fringe image is determined based on the location and the size of the calibration plate image, and all processed moiré fringe images are cropped from the captured image.

[0044] Step S34 includes:

[0045] in This refers to the initial displacement data, i.e., the grating motion data; Indicates the grating period; This represents the phase difference of the moiré fringes.

[0046] Step S4 includes: S41: Extract a moiré fringe image with fixed position reference marks from the moiré fringe image obtained in step S32; S42: The template matching method calculates the phase of the moiré fringes in a moiré fringe image with fixed position reference marks to obtain the phase difference of the moiré fringes; S43: By using the phase difference of the moiré fringes and combining it with the sampling moiré method, the phase difference of the moiré fringes is converted into the relationship between the displacement and the displacement, resulting in sub-pixel-level image displacement caused by the drone's motion. S44: Subtract the initial displacement data from the subpixel-level image displacement to obtain the displacement data of the large facility under test, thus completing the subpixel-level displacement compensation.

[0047] As one example, moiré fringes are a visual phenomenon caused by differences in the spatial frequency distribution of objects. When two objects differ in spatial frequency, interference and superposition effects occur, resulting in visually appearing striped patterns, i.e., moiré fringes. Due to their precise interference detection capabilities, they are used as tools for machining inspection of precision equipment and for quality inspection of workpiece surfaces.

[0048] As one embodiment, the sampled moiré method collects and analyzes deformation and displacement information based on moiré patterns (such as moiré fringes). The moiré fringe pattern visualized by the sampled moiré method is as follows: Figure 1 As shown. Figure 1 The point in (a) is the center position of the camera sampling point, and the number of sampling points is determined by the camera resolution; Figure 1 (b) is a grating pattern for a horizontal periodic grating, where the grating spacing is... Figure 1 (a) The sampling interval is 4.5 times that in the image. The image recorded by the camera is as follows: Figure 1 As shown in (c), the moiré pattern cannot be discerned at this point. However, if from... Figure 1 In (c), selecting one pixel every N pixels (N=4 in this case) yields... Figure 1 The moiré fringe pattern shown in (d) represents the sparsification of the image, where N is the sparsification exponent. If a second sampling point is selected and samples are taken from every N pixels, the result is as follows: Figure 1 (e) shows Phase-shifted moiré fringe images. The moiré fringe images obtained by selecting the third and fourth sampling points in the same manner are shown below. Figure 1 As shown in (f) and 1(g), this process corresponds to the phase shift of the moiré fringe pattern. If for Figure 1 All sampled (i.e., sparsified) images displayed in (dg) are interpolated using neighboring sampled data, making the images clearer and easier to observe. Figure 1 (hk) displays from Figure 1 The image obtained by linear interpolation of two neighboring sampled data (dg). Through the above steps, from... Figure 1 (c) shows multiple phase-shifted moiré fringe images.

[0049] exist Figure 1 In the middle, the first A phase shift image can be roughly represented as follows:

[0050]

[0051] in, It represents the background intensity in an image and is not sensitive to phase changes; The amplitude representing the intensity of the fringes is sensitive to changes in the phase of the fringes; It is the initial phase value of the moiré fringes.

[0052] The phase distribution of the moiré fringe pattern can be obtained using the Discrete Fourier Transform (DFT) algorithm, through formula (2):

[0053] Phase of Moiré fringes It is the phase of the sample grating Phase with reference grating The difference between them; Given a reference grating phase Analyzing the phase of moiré fringes under certain conditions The phase of the sample grating can be calculated. : The principle of displacement measurement using the sampling moiré method is as follows: Figure 2-4 As shown. The cosine intensity of the grating before and after displacement is as follows. Figure 2 As shown, the grating moves a distance in the horizontal direction. , where p is the grating period.

[0054] In one embodiment of this application, the moiré fringe image obtained by sampling difference before and after displacement is as follows: Figure 3 As shown.

[0055] The period of the moiré fringe is defined as Q, which is the period of the grating when it is displaced in the horizontal direction. hour, Figure 4 The moiré fringes in the image have shifted relative to their initial positions. The stripe motion U will cause the grating motion. Enlarged times, of which, Defined as the amplification factor.

[0056] If the grating moves in the horizontal direction Then the moiré fringes shift. The phase value of the moiré fringes will change according to the displacement of the specimen. ,but

[0057] The phase value before displacement and the phase value after displacement are respectively defined as follows: and Displacement of the sample grating It can be written in the following form:

[0058] in This refers to the initial displacement data, i.e., the grating motion; Indicates the period of the moiré fringe; Indicates the grating period; Indicates the motion of moiré fringes; This represents the phase difference of the moiré fringes; This represents the phase difference of the sample grating.

[0059] In one embodiment of this application, the following issues need to be considered when using visual methods for displacement measurement: ① Selection of shooting position: Under the premise of fixed camera parameters, the selected shooting position needs to clearly and completely present the displacement of the structure to be measured; ② The influence of factors such as lighting and environment on image quality: Conventional visual methods are more sensitive to lighting conditions, which affects the measurement results; ③ The accuracy that the selected image processing method can achieve: Since the structural displacement is usually small, generally at the millimeter level or below, the accuracy requirements of the measurement method are high.

[0060] This invention utilizes drone-based video recording, allowing for flexible selection of shooting positions and ensuring clear and complete representation of the displacement of the structure under test. During video recording of the target structure, although the drone is hovering, factors such as wind and environmental disturbances can cause slight displacements, affecting the image quality and displacement measurement results. Accurate compensation for drone motion is a key issue this invention addresses. Therefore, this invention employs an active balancing compensation method combined with the sampled Moiré method to compensate for drone displacement, ensuring the accuracy of the displacement measurement results. This invention selects the sampled moiré method as the displacement measurement method, which has advantages such as accuracy, robustness, low cost, and high speed. The sampled moiré method uses phase shift analysis to analyze the phase distribution of the moiré fringe pattern, which can obtain high-precision deformation measurement. Moreover, the phase value is obtained by analyzing each image before and after deformation, and it is not sensitive to changes in lighting conditions. In the experiment, only a grating sheet and an ordinary CCD camera are needed, and the measurement cost is far lower than other methods. Image processing only requires one image, and the required computation time is short.

[0061] When applying the sampled moiré method for displacement measurement, one of the key factors affecting measurement accuracy is the calculation of the moiré fringe shift phase. Ideally, the intensity (calculated as grayscale values) image of the moiré fringes approximates a sinusoidal image, such as... Figure 10 As shown. However, in actual shooting, due to the influence of experimental equipment and lighting, the grayscale values ​​of the moiré fringes obtained from image processing usually differ significantly from the sinusoidal image, such as... Figure 11 As shown, the image representation error in the phase calculation formula (1) is relatively large, affecting the phase calculation result. To address this problem, this invention proposes a template matching method for phase calculation, ensuring the accuracy of the phase calculation.

[0062] In engineering applications, it is often necessary to measure multiple points simultaneously, requiring the simultaneous location and capture of multiple moiré fringe images. The moiré fringe image is obtained by sampling the difference between the calibration plate image and the moiré fringe image. Therefore, the calibration plate image and the moiré fringe image have the same position and size before and after sampling difference processing. The position of the calibration plate image before sampling difference can be used to locate the moiré fringe image. This invention combines the maximum cross-correlation method to locate the coordinates of the center point of the moiré fringe image by indexing the peak value of the correlation coefficient.

[0063] The effect of using the maximum cross-correlation method to locate moiré fringes is as follows: Figure 12 As shown, the maximum cross-correlation method can only locate the center of a single image. To address this problem, this invention uses the captured image as the target image and the calibration board image as the template image. It obtains the correlation coefficient of each pixel in the target image by traversing the template image, and performs three-dimensional peak finding processing on the correlation coefficient data. The peak positions are then determined by combining the pixel size of the calibration board image: when the interval between adjacent peak positions is less than the length of the calibration board, the peak position with the larger correlation coefficient is retained as the center point coordinate, while the position obtained from the index of the smaller peak is discarded, ensuring that the obtained coordinate data is the center coordinate of the calibration board. This method can simultaneously obtain the center coordinates of multiple stripe images, thereby determining multiple stripe positions and completing image cropping, such as... Figure 13 As shown, this method enables displacement measurement of multiple points to be measured.

[0064] The effects of the technical solution in this application are as follows: 1. Theoretical image displacement Ideal calibration plate image as Figure 14 As shown, the raster period is 4.5 pixels. The calibration board image is shifted downwards by 1 pixel unit, as follows: Figure 14 As shown. With a sparsity exponent N=4, i.e., selecting one pixel every four pixels from the original ideal calibration board image at a sampling interval of one pixel, downsampling is performed. All sampled (i.e., sparsified) images are interpolated using neighboring sampled data, resulting in four sets of phase-shifted moiré fringe images, as shown. Figure 15 As shown.

[0065] According to formulas (1) to (6), the phase difference is 2 / 9π, which corresponds to a displacement of 1 pixel in the calibration plate image, consistent with the set translation distance.

[0066] This application also discloses an electronic device. (See reference...) Figure 16 , Figure 16 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0067] The communication bus 502 is used to enable communication between these components.

[0068] The user interface 503 may include a display screen, and optionally, the user interface 503 may also include a standard wired interface or a wireless interface.

[0069] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0070] This application also discloses a computer-readable storage medium storing multiple instructions adapted for loading by a processor to execute the above-described method for dynamic visual displacement measurement of large facilities based on unmanned aerial vehicles.

[0071] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure.

[0072] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for measuring the dynamic visual displacement of large facilities based on unmanned aerial vehicles (UAVs), characterized in that, The method includes the following steps: S1: Collect UAV images of the large facility under test before and after displacement; S2: Perform motion compensation on drone images; S3: Based on the sampling moiré method and the maximum cross-correlation method, the motion-compensated UAV imagery is processed to obtain the initial displacement data; S4: Based on the sampling moiré method and the maximum cross-correlation method, combined with the initial displacement data, subpixel-level motion compensation is performed on the UAV imagery to complete the displacement measurement of the large facility under test.

2. The method for measuring the dynamic visual displacement of large facilities based on unmanned aerial vehicles (UAVs) as described in claim 1, characterized in that, Step S1 includes: S11: Affix a set of markers to the large facility to be tested; A fixed position reference mark is selected at a location in the large facility under test where no structural displacement occurs. There are two fixed position reference markers; Mark the points to be tested in the large facility to be tested; S12: Take drone images of the large facility under test before and after deformation using a drone; The UAV imagery includes: fixed position reference markers and markers for the points to be measured.

3. The method for measuring the dynamic visual displacement of large facilities based on unmanned aerial vehicles (UAVs) as described in claim 1, characterized in that, Step S2 includes: S21: Locate the fixed position reference marker in the UAV imagery, and determine the center coordinates and reference line; S22: Use the initial frame image in the UAV imagery as the reference image, and the other images as the target images; S23: Simplify the six degrees of freedom motion of the UAV to four degrees of freedom, which includes: motion along the x-axis. Motion along the y-axis Rotation angle and image scaling The four degrees of freedom are mapped to the translation, rotation, and scaling transformations of UAV imagery. S24: Align the target image with the reference image using a similarity transformation algorithm, combined with the center coordinates and reference lines.

4. The method for measuring the dynamic visual displacement of large facilities based on unmanned aerial vehicles (UAVs) as described in claim 1, characterized in that, Step S3 includes: S31: Extract keyframe images from UAV imagery; S32: Scale up or down the keyframe image proportionally to generate moiré fringes; By downsampling and intensity interpolation, multiple phase-shifted moiré fringes of the keyframe image are obtained to obtain the moiré fringe image; S33: The template matching method is used to calculate the phase of the moiré fringes in the moiré fringe image to obtain the phase difference of the moiré fringes; S34: By using the phase difference of the moiré fringes and combining it with the principle of sampling moiré, the phase difference of the moiré fringes is converted into the relationship between the displacement, and the initial displacement data is obtained.

5. The method for measuring the dynamic visual displacement of large facilities based on unmanned aerial vehicles (UAVs) as described in claim 4, characterized in that, Step S33 includes: A complete cycle of moiré fringe image is constructed using the moiré fringe image as a template image; The moiré fringe images before and after displacement are matched with the template image respectively; By combining the maximum cross-correlation method, the center point coordinates of the moiré fringe image are located by indexing the peak values ​​of the correlation coefficients. The phase difference of the moiré fringes is obtained by calculating the shift of the center point coordinates.

6. The method for measuring the dynamic visual displacement of large facilities based on unmanned aerial vehicles (UAVs) as described in claim 4, characterized in that, Step S34 includes: in This refers to the initial displacement data, i.e., the grating motion data; Indicates the grating period; This represents the phase difference of the moiré fringes.

7. The method for measuring the dynamic visual displacement of large facilities based on unmanned aerial vehicles (UAVs) as described in claim 4, characterized in that, Step S4 includes: S41: Extract a moiré fringe image with fixed position reference marks from the moiré fringe image obtained in step S32; S42: The template matching method calculates the phase of the moiré fringes in a moiré fringe image with fixed position reference marks to obtain the phase difference of the moiré fringes; S43: By using the phase difference of the moiré fringes and combining it with the sampling moiré method, the phase difference of the moiré fringes is converted into the relationship between the displacement and the displacement, resulting in sub-pixel-level image displacement caused by the drone's motion. S44: Subtract the initial displacement data from the subpixel-level image displacement to obtain the displacement data of the large facility under test, thus completing the subpixel-level displacement compensation.

8. An electronic device, characterized in that, The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed by a computer, perform the method as described in any one of claims 1-7.

Images