Unmanned aerial vehicle interference weakening method for stay cable displacement visual detection

By capturing cable vibration videos with drones and utilizing computer vision algorithms and camera calibration technology, drone interference is reduced, and the vibration frequency of the cable is accurately identified. This solves the problem of drone vibration affecting the detection results and improves the accuracy and automation of cable displacement detection.

CN118918081APending Publication Date: 2024-11-08ZHENGZHOU UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410968133.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

When drones film cable vibrations, their own vibrations interfere with the accuracy of cable displacement detection, resulting in inaccurate detection results and making it difficult to automatically identify the various modal frequencies of the cable.

Method used

The vibration video of the cable was captured by a drone, and the vibration video of the cable was analyzed by computer vision algorithms. The pose changes of the drone were decomposed by improving the region growing algorithm and planar homography matrix decomposition. The object distance was calculated by combining the pinhole imaging principle and camera calibration. After noise reduction, the displacement frequency of the cable was extracted.

Benefits of technology

This effectively reduces the interference of the drone's own movement on cable vibration, improves the automation level of cable vibration frequency identification, and reduces detection errors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918081A_ABST
    Figure CN118918081A_ABST
Patent Text Reader

Abstract

The invention discloses an unmanned aerial vehicle interference weakening method for stay cable displacement visual detection, and the method comprises the steps: firstly shooting a vibration video of a cable through an unmanned aerial vehicle, carrying out the edge detection of the cable through a computer vision algorithm, and calculating the mixed displacement data of the unmanned aerial vehicle and the cable; afterwards, in order to reduce the vibration influence of the unmanned aerial vehicle and obtain a frequency spectrum image with a high signal-to-noise ratio, the pose change of the unmanned aerial vehicle is obtained by decomposing a plane homography matrix; then distance information between the unmanned aerial vehicle and the cable is obtained by using an improved region growing algorithm; and finally, aligning a camera coordinate system of a subsequent frame with a coordinate system of the first frame by using the information, thereby obtaining cable displacement information after noise reduction, transforming data into frequency after Fourier transform, and obtaining vibration frequency of each order of the cable by using a Peak Pickering algorithm. The cable vibration frequency detection precision of the unmanned aerial vehicle is improved, and the influence of the motion of the unmanned aerial vehicle is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of bridge structure health monitoring, and in particular to a method for reducing interference of unmanned aerial vehicles used for visual detection of cable-stayed cable displacement. Background Art

[0002] Cables and suspenders are important components of the load-bearing structure of long-span bridges. Accurately measuring their vibration frequency is of great significance for cable force detection and damage assessment. At present, the vibration measurement methods of cables can be divided into two categories: one is the monitoring method in which the sensor contacts the cable, and the other is the non-contact method in which the sensor does not contact the cable. The first type of contact monitoring method mainly uses accelerometers for monitoring. This method requires the accelerometer to be fixed on the surface of the cable, and then connected to the data acquisition instrument through a connecting line. The vibration frequency of the cable is obtained by analyzing the acceleration time history of the cable vibration. However, due to the long distance and large number of cable structures, the health monitoring system of the contact monitoring method is expensive. The other type of method is non-contact monitoring method, which mainly includes laser Doppler technology, global positioning system (GPS), microwave remote sensing and computer vision. However, this type of method often requires professional instruments and equipment, which is costly and has low efficiency in detecting all cables.

[0003] In recent years, due to the rapid development of drones, their application in the field of civil engineering structural health monitoring and detection has gradually increased, such as structural surface damage identification and structural displacement detection. Similarly, drones can also be used in the field of cable structure vibration frequency detection. Since drones need to hover in the air when shooting the vibration frequency of cable structures, they are easily affected by environmental winds, and their three-dimensional positions in the air are constantly changing. Although the drone's camera has a mechanical stabilization device, the device still cannot completely compensate for the drone's own movement, causing the cable vibration displacement time history detected by it to include two parts: cable vibration and drone's own vibration. Therefore, the drone's own vibration will affect the time history results of cable displacement detection, resulting in inaccurate detection results and difficulty in automatically identifying the various modal frequencies of the cable.

[0004] In view of the above, the present application provides a method for reducing interference from drones for visual detection of cable-stayed cable displacement to solve the above problems. Summary of the invention

[0005] In view of the above situation, the present invention provides a method for reducing interference from a drone for visual detection of displacement of a stay cable, which is used to solve the problem of mutual interference between the cable and the drone vibration.

[0006] A method for reducing interference from unmanned aerial vehicles for visual detection of stay cable displacement, comprising:

[0007] Image acquisition: Video of cable vibration captured by drone;

[0008] Position recognition; using the improved region growing algorithm in computer vision algorithm to analyze the vibration video of the cable and calculate the mixed displacement data of the drone and the cable;

[0009] Eliminate the influence; obtain the posture change of the drone (camera) by decomposing the plane homography matrix;

[0010] Object distance calculation: The pixel focal length of the camera is obtained through camera calibration, and the pixel width of each frame of the cable is obtained by using the improved region growing algorithm in the computer vision algorithm. Finally, the object distance information is calculated by combining the pinhole imaging principle.

[0011] Camera posture calculation: obtain the feature points that match the subsequent frame image with the first frame image through the feature point matching algorithm, then calculate the plane homography matrix, and then decompose it into the camera rotation and displacement matrix of the subsequent frame with respect to the first frame;

[0012] Cable displacement denoising: Use the object distance information to convert the position information of the subsequent frame cable in the imaging plane to the camera coordinate system, then use the camera posture change to align the subsequent frame camera coordinate system with the first frame, and then convert it to the imaging plane, and calculate the cable displacement after denoising in a loop;

[0013] Frequency extraction: Perform Fourier transform on the cable displacement after noise reduction, and then use the PeakPicking algorithm to obtain the frequencies of each order of the cable.

[0014] The above technical solution has the following beneficial effects:

[0015] The present invention provides a method for reducing interference from drones for visual detection of displacement of a stay cable. The method uses a drone to quickly shoot a vibration video of the cable. After analysis by a computer vision algorithm, the vibration frequency of the cable can be better identified without recording the movement of the drone itself, thereby reducing the interference of the drone's own movement on the cable vibration and improving the degree of automation in identifying the cable vibration frequency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the process of the present invention;

[0017] Figure 2 A schematic diagram comparing the effects of noise reduction before and after the X-direction in an embodiment of the present invention;

[0018] Figure 3 A schematic diagram comparing the effects of noise reduction before and after Y direction reduction in an embodiment of the present invention;

[0019] Figure 4 Schematic diagram of the frequency domain before and after noise reduction in an embodiment of the present invention;

[0020] Figure 5It is a schematic diagram of the principle (object distance calculation) of pinhole imaging of the present invention;

[0021] Figure 6 Schematic diagram of the position relationship of the camera coordinate system between the first frame and subsequent frames of the present invention. DETAILED DESCRIPTION

[0022] The aforementioned and other technical contents, features and effects of the present invention will be clearly presented in the following detailed description of the embodiments with reference to the drawings of the present application. The structural contents mentioned in the following embodiments are all based on the drawings of the specification as reference.

[0023] The present invention provides a method for reducing interference of unmanned aerial vehicles for visual detection of displacement of a stay cable, comprising:

[0024] Step S1: Use a drone to capture a vibration video of the cable.

[0025] Step S2: Analyze the vibration video of the cable using the improved region growing algorithm in the computer vision algorithm to calculate the mixed displacement data of the drone and the cable.

[0026] Step S3: object distance calculation: the pixel focal length of the camera is obtained by camera calibration, and then the pixel width of each frame of the cable is obtained by using the improved region growing algorithm. Finally, the object distance information is calculated in combination with the pinhole imaging principle to obtain the distance information between the drone and the cable.

[0027] The relationship between the distance from the optical center to the object and the object size (diameter) is shown in formula 1.1, which can be obtained by camera calibration. Through formula 1.1, combined with the pixel width of the inclined cable obtained by the above method, the present application can calculate the distance between the drone and the inclined cable in each frame image.

[0028]

[0029] Where d is the diameter of the cable in the imaging plane, D is the actual diameter of the cable, f is the focal length in pixels, and U represents the object distance.

[0030] Step S4: Calculate the camera posture; obtain the feature points of the subsequent frame image matching with the first frame image through the feature point matching algorithm, then calculate the plane homography matrix, and then decompose it into the camera rotation and displacement matrix of the subsequent frame with respect to the first frame;

[0031] The camera attitude calculation uses the intrinsic parameter matrix and the matched feature points in the background to calculate the camera attitude. The camera attitude calculation plays an important role in the next step of displacement denoising, because in displacement denoising, it is necessary to use the attitude change to align the camera coordinate system of subsequent frames with the first frame to achieve the purpose of denoising.

[0032] To calculate the change in the camera's posture, we first calculate the homography matrix, which is also called the homogeneous transformation matrix. It is a tool used in computer vision and image processing to describe the correspondence between points in two planes. We can establish a correspondence between p and p' as shown in Formula 1.2, where the coordinates of p are (u, v) and the coordinates of p' are (u', v'). p and p' represent the coordinates of the point P in space projected onto the imaging plane before and after the camera (drone) moves. The matrix H is a 3×3 matrix containing 9 parameters. These parameters define the transformation from one perspective to another. This transformation takes into account the internal parameters of the camera (such as focal length, aperture, distortion, etc.), external parameters (such as position, orientation), and the relative position and posture between the two imaging planes.

[0033]

[0034] The internal parameter matrix K composed of the internal parameters of the camera is essential for subsequent processing. This matrix contains important parameters such as focal length and optical axis center point coordinates, which is crucial for subsequent image processing and analysis. The composition of K is shown in formula 1.3, where f x and f y are the focal lengths of the camera in the x-axis and y-axis directions, respectively, and u0 and v0 are the coordinates of the principal point of the image (usually the center of the image).

[0035]

[0036] This process can be completed with the help of MATLAB's built-in camera calibration function. The specific operation method is to take at least 10 chessboard pictures at different angles (camera calibration requires a chessboard as a calibration object to calculate the camera's intrinsic parameter matrix). MATLAB can accurately calculate the camera's intrinsic parameter matrix K by analyzing these pictures.

[0037] The plane homography matrix contains the information of rotation and translation between the two images, that is, the change in the camera's posture. After obtaining the camera's intrinsic parameter matrix, the camera's rotation matrix R and displacement matrix T can be derived by formulas 1.4-1.10.

[0038]

[0039] λ=1 / ||K -1 h1|| (1.5)

[0040] r1=λK -1 h1 (1.6)

[0041] r2=λK -1 h2 (1.7)

[0042] r3=r1×r2 (1.8)

[0043] t=λK -1 h3 (1.9)

[0044] R=[r1 r2 r3] (1.10)

[0045] Among them, r1 and r2 represent the first two columns of the rotation matrix R, t is the displacement matrix, h1, h2, and h3 represent the first, second, and third columns of the plane homography matrix, and λ has no actual physical meaning and is an intermediate value in the calculation process.

[0046] Step S5: cable displacement denoising; using the object distance information, the position information of the cable in the subsequent frames in the imaging plane is converted into the camera coordinate system, and then the camera coordinate system of the subsequent frames is aligned with the first frame using the camera posture change, and then converted to the imaging plane, and the cable displacement after denoising is obtained by cyclic calculation;

[0047] Formula 1.11 describes in detail the coordinate position relationship between the point on the imaging plane, the point in the camera coordinate system, and the corresponding point P in the world coordinate system (representing point P in the world coordinate system). This relationship is the key to understanding how the camera maps the three-dimensional world to a two-dimensional image, where the coordinates of p are (u, v)Z c The calculation is performed as shown in formula 1.5, where K is the camera's intrinsic parameter matrix, as shown in formula 1.3.P C (X, Y, Z) is the position information of P in the world coordinate system in the camera coordinate system.

[0048]

[0049] Then, using the camera coordinate system as a reference, the decomposed camera rotation and translation information is used to calculate the displacement information of the cable when the camera is stationary. The specific derivation is as follows:

[0050] Assume that the coordinates of the projection point p of point P in the first frame on the two-dimensional plane are (u, v), and the coordinates of the projection point p' of point P in a subsequent frame on the imaging plane are (u', v'). First, according to formula 1.12, the two-dimensional position information of point p' in the subsequent frame is converted to the camera space coordinate system to obtain P C (X, Y, Z). U is the distance from the optical center to point P. This information is very important for converting two-dimensional coordinates to three-dimensional coordinates. It is obtained according to formula 1.1. Finally, the three-dimensional coordinates are projected onto the imaging plane of the first frame by formula 1.13.

[0051]

[0052] The camera coordinate system position relationship between the first frame and subsequent frames is as follows Figure 6As shown, using the previously obtained R and T matrices, the P coordinates (X, Y, U) of the subsequent frames are transformed into the camera coordinate system of the first frame through formula 1.13. At this time, the coordinates of the subsequent frame point P in this camera coordinate system are (X', Y', Z'). The displacement data of the inclined cable can be calculated by difference. Formula 1.14 can transform the two-dimensional information (u', v') into the camera coordinate system, which provides a basis for further analysis and processing. The position information of the inclined cable in the imaging plane has been obtained in step S2. The camera intrinsic parameter matrix can be obtained by camera calibration. The intrinsic parameter matrix is ​​a matrix composed of the intrinsic parameters of a camera, which contains pixel focal length information.

[0053] Step S6: Modal frequency extraction; perform Fourier transform on the displacement of the cable after denoising in step S5, and then use the Peak Picking algorithm to obtain the frequencies of each order of the cable. As shown in Table 1.1, the proposed method can effectively reduce the energy of the data below 0.5Hz, reducing it by 71.81%. This means that the proposed method can effectively reduce the strong low-frequency interference of the drone in the mixed displacement data. Table 1.2 records the modal frequencies obtained by the proposed method and compares them with the modal frequencies obtained by the accelerometer as a reference value. Among them, the average absolute error is 0.0227Hz, and the average relative error is 1.02%.

[0054] Table 1.1 Energy calculation of 0-0.5Hz before and after compensation

[0055] area Raw data After noise reduction Noise reduction effect Energy from 0 to 0.5 Hz 5.2546e+12 1.4812e+12 71.81% Total Energy 5.2654e+12 1.4858e+12 ——

[0056] Table 1.2 Modal frequency comparison

[0057]

[0058] The above description is only for illustrating the present invention. It should be understood that the present invention is not limited to the above embodiments, and various variations that conform to the concept of the present invention are within the protection scope of the present invention.

Claims

1. A method for reducing interference from unmanned aerial vehicles for visual detection of cable displacement, characterized in that: include: Image acquisition: Video of cable vibrations captured by drone; Location identification; The improved region growing algorithm in computer vision algorithm is used to analyze the vibration video of the cable and calculate the mixed displacement data of the drone and the cable; Eliminate the influence; obtain the posture change of the drone by decomposing the plane homography matrix; Calculate the object distance; use the improved region growing algorithm in the computer vision algorithm to obtain the pixel width of each frame of the cable, and finally calculate the object distance information based on the pinhole imaging principle; Camera posture calculation: obtain the feature points that match the subsequent frame image with the first frame image through the feature point matching algorithm, then calculate the plane homography matrix, and then decompose it into the camera rotation and displacement matrix of the subsequent frame with respect to the first frame; Cable displacement noise reduction; The position information of the cable in the subsequent frames in the imaging plane is converted to the camera coordinate system using the object distance information, and then the camera coordinate system of the subsequent frames is aligned with that of the first frame using the camera posture change, and then converted to the imaging plane, and the displacement of the cable after noise reduction is obtained by cyclic calculation; Modal frequency extraction: The cable displacement after noise reduction is Fourier transformed, and then the PeakPicking algorithm is used to obtain the frequencies of each order of the cable.

2. The method for reducing interference of unmanned aerial vehicles for visual detection of displacement of stay cables according to claim 1 is characterized in that: The feature point matching algorithm is an ORB algorithm, and matching feature points are extracted from two input images.

3. The method for reducing interference of unmanned aerial vehicles for visual detection of cable displacement according to claim 1 is characterized in that: When calculating the object distance, the camera’s pixel focal length and intrinsic parameter matrix are first obtained through camera calibration.

4. The method for reducing interference of unmanned aerial vehicles for visual detection of cable displacement according to claim 1 is characterized in that: The PeakPicking algorithm is used to identify the peak abscissa on the frequency domain graph.