Bridge cable vibration measurement method and device based on dual-vision fusion of unmanned aerial vehicle

Abstract

The invention discloses a bridge cable vibration measurement method and device based on unmanned aerial vehicle dual-vision fusion. The method comprises the following steps: acquiring a vibration video of a bridge fixed target and a vibration video of a bridge cable which are respectively acquired by a wide-angle camera and a long-focus camera which are connected to an unmanned aerial vehicle, and performing self-vibration elimination processing of the unmanned aerial vehicle on the vibration video of the bridge cable by utilizing homography transformation according to the vibration video of the bridge fixed target, and performing segmentation processing on the absolute vibration video of the bridge cable by using a trained neural network segmentation model to obtain the absolute vibration video of the bridge cable after segmentation processing, and extracting displacement from the absolute vibration video of the segmented bridge cable by using a sub-pixel edge detection algorithm to obtain the full-field displacement of the bridge cable, and obtaining the full-field vibration mode of the bridge cable by using a data-driven random subspace recognition method according to the full-field displacement of the bridge cable.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of bridge health monitoring, and particularly relates to a bridge cable vibration measurement method and device based on unmanned aerial vehicle dual vision fusion. BACKGROUND

[0002] As an important means of bridge structure safety assessment, health monitoring technology based on vibration response can effectively obtain key information such as cable deformation, modal parameters and cable force by measuring the vibration response of the cable, and provides an important basis for bridge health condition assessment. However, the traditional contact type sensor has the disadvantages of high equipment cost, complex installation, limited measuring points and the like, and it is difficult to meet the needs of real-time monitoring of dynamic response of long-span bridge cables. In recent years, the rapid development of computer vision technology has provided a non-contact solution for bridge health monitoring. This method has shown great application potential in the field of bridge health monitoring due to its low cost, simple operation and multi-point measurement advantages.

[0003] At present, although the vibration measurement method based on computer vision has made certain progress, it still faces the following technical bottlenecks in practical application. First, most of the existing visual methods rely on the installation of artificial targets on the surface of the cable, which not only increases the complexity and cost of target installation, but also makes it very difficult to install targets in high-altitude or hard-to-reach areas, which seriously limits its application range. Secondly, long-span bridges have the characteristics of large span and wide bridge site water area, and the traditional computer vision measurement method is difficult to carry out close-range displacement measurement, and the camera imaging is easily affected by the lens tilt angle, atmospheric turbulence and other errors. In addition, the installation process of fixed cameras is complex and lacks flexibility, and it is difficult to adapt to the complex test environment of long-span bridges. In contrast, the unmanned aerial vehicle platform can effectively overcome the limitations of fixed cameras due to its high efficiency, flexibility and high-definition imaging capability. The unmanned aerial vehicle platform provides a new technical path, which can improve the accuracy and reliability of bridge cable vibration measurement, especially in the complex monitoring environment of long-span bridges, and has broad application prospects.

[0004] The bridge displacement measurement method based on the unmanned aerial vehicle has broad application prospects in structural health monitoring, and effectively solves the problem that the measurement precision of the fixed camera is limited at a long distance, and provides a new idea for vibration measurement of long-span bridges. However, the unmanned aerial vehicle inevitably produces vibration during flight, thereby affecting the measurement precision of the displacement, especially in the case of using a long-focus lens, a small camera movement can cause a large measurement error. Although the existing technology has made certain progress in vibration measurement based on the unmanned aerial vehicle, most methods still rely on a fixed reference point to eliminate the vibration of the unmanned aerial vehicle. In the complex test environment of long-span bridges, there are problems such as difficulty in selecting a fixed reference point, incomplete extraction of structural vibration information, and great influence of light conditions, and further research on the vibration elimination method of the unmanned aerial vehicle is still needed.

[0005] Therefore, how to provide a scheme capable of solving the above technical problems is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0006] To solve the above technical problems, the present application provides a bridge cable vibration measurement method, device, equipment and medium based on unmanned aerial vehicle dual vision fusion, which can eliminate the influence of the vibration of the unmanned aerial vehicle on the bridge vibration measurement results, thereby improving the accuracy and practicality of the vibration measurement technology based on the unmanned aerial vehicle.

[0007] In one aspect, the present application provides a bridge cable vibration measurement method based on unmanned aerial vehicle dual vision fusion, comprising: obtaining vibration videos of bridge fixed targets and vibration videos of bridge cables collected by a wide-angle camera and a long-focus camera connected to the unmanned aerial vehicle, wherein the wide-angle camera and the long-focus camera are arranged on different axes; According to the vibration video of the bridge fixed target, the vibration of the unmanned aerial vehicle is eliminated by using homographic transformation on the vibration video of the bridge cable to obtain the absolute vibration video of the bridge cable; The trained neural network segmentation model is used to segment the absolute vibration video of the bridge cable to obtain the segmented absolute vibration video of the bridge cable; The sub-pixel edge detection algorithm is used to extract the displacement from the segmented absolute vibration video of the bridge cable to obtain the full-field displacement of the bridge cable; According to the full-field displacement of the bridge cable, the data-driven random subspace identification method is used to obtain the full-field mode shape of the bridge cable.

[0008] Preferably, according to the vibration video of the bridge fixed target, the vibration of the unmanned aerial vehicle is eliminated by using homographic transformation on the vibration video of the bridge cable to obtain the absolute vibration video of the bridge cable, comprising: obtain an intrinsic matrix of the wide-angle camera and an intrinsic matrix of the long-focus camera; According to the vibration video of the bridge fixed target, the 3D pose of the wide-angle camera at each time step is calculated; According to the intrinsic matrix of the wide-angle camera, the intrinsic matrix of the long-focus camera, and the 3D pose of the wide-angle camera at each time step, the rotation matrix and the translation vector of the wide-angle camera are calculated; According to the rotation matrix and the translation vector of the wide-angle camera, the Euclidean homography matrix of the wide-angle camera is calculated; According to the intrinsic matrix of the long-focus camera and the Euclidean homography matrix of the wide-angle camera, the projection homography matrix of the long-focus camera is calculated; According to the projection homography matrix of the long-focus camera, the vibration video of the bridge cable is processed to eliminate the vibration of the unmanned aerial vehicle itself, so as to obtain the absolute vibration video of the bridge cable.

[0009] Preferably, according to the rotation matrix and the translation vector of the wide-angle camera, the Euclidean homography matrix of the wide-angle camera is calculated, including: The Euclidean homography matrix of the wide-angle camera is calculated by the following formula:

[0010] wherein, H e represents the Euclidean homography matrix of the wide-angle camera, represents the rotation matrix of the wide-angle camera, represents the translation vector of the wide-angle camera, n T represents the transpose matrix of the unit normal vector of the bridge fixed target plane in the first camera coordinates, d represents the distance from the wide-angle camera to the bridge fixed target plane.

[0011] Preferably, according to the intrinsic matrix of the long-focus camera and the Euclidean homography matrix of the wide-angle camera, the projection homography matrix of the long-focus camera is calculated, including: The projection homography matrix of the long-focus camera is calculated by the following formula:

[0012] wherein, H p represents the projection homography matrix of the long-focus camera, K t represents the intrinsic matrix of the long-focus camera, represents K t the inverse matrix of

[0013] Preferably, a sub-pixel edge detection algorithm is used to extract displacement from the segmented absolute vibration video of the bridge cables to obtain the full-field displacement of the bridge cables, including: Pixel-level edge points of bridge cables are extracted from the segmented absolute vibration video of bridge cables using an edge detection algorithm; Subpixel detection is performed on each pixel-level edge point using a preset detection box, and the subpixel coordinates of each pixel-level edge point are calculated using the local area effect to obtain the subpixel-level edge points of the bridge cable. An edge tracking algorithm is used to track the sub-pixel level edge points of the bridge cables in order to obtain the full-field displacement of the bridge cables.

[0014] Preferably, before extracting pixel-level edge points of the bridge cables from the segmented absolute vibration video of the bridge cables using an edge detection algorithm, the method further includes: The absolute vibration video of the bridge cables after segmentation was smoothed using a Gaussian smoothing kernel.

[0015] Preferably, the full-field vibration modes of the bridge cables are obtained using a data-driven random subspace identification method based on the full-field displacement of the bridge cables, including: Construct the Hankel matrix based on the total field displacement; The Hankel matrix is ​​transformed by projection to obtain the projection matrix; The projection matrix is ​​subjected to QR decomposition and singular value decomposition, and the eigenvalues ​​and eigenvectors are calculated using the least squares method. Based on eigenvalues ​​and eigenvectors, the full-field vibration modes of the bridge cables are obtained.

[0016] Another aspect of the present invention provides a bridge cable vibration measurement device based on UAV dual-vision fusion, comprising: The acquisition module is used to acquire vibration videos of the fixed bridge target and the bridge cables, respectively, through a wide-angle camera and a telephoto camera connected to the drone. The wide-angle camera and the telephoto camera are set to be out of axis. The vibration cancellation module is used to perform self-vibration cancellation processing on the vibration video of the bridge cables based on the vibration video of the fixed target of the bridge, using homography transformation, so as to obtain the absolute vibration video of the bridge cables. The segmentation processing module is used to segment the absolute vibration video of the bridge cable using a trained neural network segmentation model to obtain the segmented absolute vibration video of the bridge cable. The displacement extraction module is used to extract displacement from the segmented absolute vibration video of the bridge cable using a sub-pixel edge detection algorithm, so as to obtain the full-field displacement of the bridge cable. The mode shape acquisition module is used to obtain the full-field mode shape of the bridge cable based on the full-field displacement of the bridge cable using a data-driven random subspace identification method.

[0017] Another aspect of the present invention provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the steps of the bridge cable vibration measurement method based on UAV dual-vision fusion as described above.

[0018] In another aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for measuring bridge cable vibration based on UAV dual-vision fusion.

[0019] The present invention has at least the following beneficial effects: This invention acquires vibration videos of a fixed bridge target and bridge cables, respectively, using a wide-angle camera and a telephoto camera connected to a drone. The wide-angle and telephoto cameras are set off-axis. Based on the vibration video of the fixed bridge target, homography transformation is used to perform drone-based vibration cancellation processing on the bridge cable vibration video to obtain the absolute vibration video of the bridge cables. A trained neural network segmentation model is then used to segment the absolute vibration video of the bridge cables to obtain the segmented absolute vibration video of the bridge cables. A sub-pixel edge detection algorithm is used to extract displacement from the segmented absolute vibration video of the bridge cables to obtain the full-field displacement of the bridge cables. Based on the full-field displacement of the bridge cables, a data-driven random subspace recognition method is used to obtain the full-field mode shape of the bridge cables. Using the solution of this application, the influence of the drone's own vibration on the bridge vibration measurement results can be eliminated, thereby improving the accuracy and practicality of drone-based vibration measurement technology. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a method for measuring bridge cable vibration based on dual-vision fusion from an unmanned aerial vehicle (UAV) according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a dual-camera unmanned aerial vehicle system provided in an embodiment of the present invention; Figure 3 A pixel point provided in an embodiment of the present invention P A schematic diagram of grayscale values; Figure 4 A schematic diagram of another sub-pixel edge detection algorithm based on local area effect provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a bridge cable vibration measurement device based on UAV dual-vision fusion, provided as an embodiment of the present invention. Detailed Implementation

[0022] The core of this invention is to provide a method, device, equipment, and medium for measuring bridge cable vibration based on UAV dual-vision fusion, which can eliminate the influence of the UAV's own vibration on the bridge vibration measurement results, thereby improving the accuracy and practicality of UAV-based vibration measurement technology.

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] One embodiment of the present invention provides a method for measuring bridge cable vibration based on UAV dual-vision fusion. Please refer to [link to relevant documentation]. Figure 1 The method includes: Step S110: Acquire vibration videos of the fixed bridge target and the bridge cables, respectively, by using a wide-angle camera and a telephoto camera connected to the drone. The wide-angle camera and the telephoto camera are set to be out of axis.

[0025] In embodiments of the present invention, such as Figure 2 As shown, a quadcopter drone can be used to carry two cameras—a wide-angle camera and a telephoto camera—connected by gimbals on different axes. The wide-angle camera captures vibration videos of fixed targets on the bridge, such as bridge piers, while the telephoto camera captures vibration videos of the bridge cables. The camera gimbals can be locked to approximately the same direction and operate synchronously. Even when the two camera gimbals are oriented in the same direction, the coordinate systems of the wide-angle and telephoto cameras are not rigidly connected or fixed. Due to the different performance characteristics of the wide-angle and telephoto cameras, their responses to drone motion differ; they show only a slight similarity in the translational direction, while exhibiting significant differences in the magnitude of translation and the rotational angle.

[0026] Step S120: Based on the vibration video of the fixed target of the bridge, the vibration video of the bridge cable is processed by the UAV to eliminate its own vibration, so as to obtain the absolute vibration video of the bridge cable.

[0027] In this embodiment of the invention, when a telephoto camera mounted on a drone collects vibration video of bridge cables, the vibration video contains not only the vibration information of the bridge cables but also the vibration information of the drone itself. However, when a wide-angle camera mounted on a drone collects vibration video of a fixed target on the bridge, the fixed target can be considered stationary, and the vibration video only contains the vibration information of the drone itself. Therefore, based on the vibration video of the fixed target on the bridge, and using homography transformation, the vibration video of the bridge cables can be processed to eliminate the drone's own vibration, so that the processed vibration video contains only the vibration information of the bridge cables. Homography transformation is a geometric transformation in computer vision used to describe the mapping relationship between two planes, and is widely used in tasks such as image registration, correction, and panoramic stitching. It is understood that this method relies only on the video data from two cameras, requires no additional sensor information (such as an inertial measurement unit), and does not require precise calibration of the coordinate system relationship between the two cameras, greatly improving the accuracy and practicality of drone-based vibration measurement technology.

[0028] Step S130: Use the trained neural network segmentation model to segment the absolute vibration video of the bridge cable to obtain the segmented absolute vibration video of the bridge cable.

[0029] In this embodiment of the invention, the vibration video of the bridge cable captured by the telephoto camera includes not only the bridge cable itself, but may also include other background elements such as buildings, tree branches, vehicles, and pedestrians. A neural network segmentation model can be pre-trained. After acquiring the vibration video of the bridge cable, the trained neural network segmentation model is used to segment the bridge cable, achieving accurate segmentation of the bridge cable in complex background images.

[0030] In practice, a telephoto camera mounted on a drone can be used to collect vibration videos of bridge cables under various complex backgrounds, creating an image dataset of bridge cables, which is then divided into training and testing sets. By combining the advantages of Convolutional Neural Networks (CNNs) and the Transformer deep learning architecture for local and global feature extraction, a neural network segmentation model, ConvTransNet, for bridge cables under complex backgrounds is constructed. The ConvTransNet neural network segmentation model is trained using data from the training set and validated using data from the testing set until its accuracy meets the requirements; at this point, it is considered a trained neural network segmentation model, ConvTransNet.

[0031] Step S140: Use a subpixel edge detection algorithm to extract displacement from the segmented absolute vibration video of the bridge cable to obtain the full-field displacement of the bridge cable.

[0032] In this embodiment of the invention, edge detection is a crucial technique for identifying physical edges and extracting image features in the field of image processing. An edge refers to a region in an image where pixel values ​​change drastically, typically corresponding to the outline of an object in a scene. Edge detection algorithms identify these boundaries by calculating pixel changes in local areas of an image. Traditional pixel-level edge detection algorithms can only determine the position of edge pixels up to the nearest integer pixel location. While simple to implement, this method's accuracy is limited by pixel size. Subpixel edge detection algorithms, however, can locate edges to subpixel positions within a pixel, significantly improving edge localization accuracy. By extracting subpixel-level edge points of bridge cables from the segmented absolute vibration video of bridge cables using a subpixel edge detection algorithm, and then tracking these subpixel-level edge points using an edge tracking algorithm, high-precision identification of the full-field displacement of the bridge cables can be achieved. It is understood that this method eliminates the need for installing manual targets on the surface of the bridge cables, reducing labor costs and avoiding limitations in application due to difficulties in installing targets at high altitudes or inaccessible areas.

[0033] Step S150: Based on the full-field displacement of the bridge cables, obtain the full-field vibration modes of the bridge cables using the data-driven random subspace identification method.

[0034] In this embodiment of the invention, the Data-Stochastic Subspace Identification (Date-SSI) method has advantages such as strong noise resistance and high identification accuracy, and has been widely used in the identification of structural mode shapes using pure output. After obtaining the full-field displacement of the bridge cables, the Data-Stochastic Subspace Identification method can automatically acquire the high-resolution full-field mode shapes of the bridge cables, and then calculate key information such as the deformation, modal parameters, and cable forces of the bridge cables, providing an important basis for assessing the health status of bridges.

[0035] As described above, by acquiring vibration videos of a fixed bridge target and bridge cables respectively, captured by a wide-angle camera and a telephoto camera connected to a UAV (with the wide-angle and telephoto cameras set off on different axes), the vibration video of the bridge cables is processed using homography transformation to eliminate the UAV's own vibration, thus obtaining the absolute vibration video of the bridge cables. A trained neural network segmentation model is then used to segment the absolute vibration video of the bridge cables, resulting in a segmented absolute vibration video of the bridge cables. A sub-pixel edge detection algorithm is used to extract displacement from the segmented absolute vibration video of the bridge cables, obtaining the full-field displacement of the bridge cables. Based on the full-field displacement of the bridge cables, a data-driven random subspace recognition method is used to obtain the full-field mode shape of the bridge cables. Using the scheme of this application, the influence of the UAV's own vibration on the bridge vibration measurement results can be eliminated, thereby improving the accuracy and practicality of UAV-based vibration measurement technology.

[0036] Optionally, in the above embodiments, step S120 includes: Step S1201: Obtain the intrinsic parameter matrix of the wide-angle camera and the intrinsic parameter matrix of the telephoto camera.

[0037] Step S1202: Calculate the 3D pose of the wide-angle camera at each time step based on the vibration video of the fixed target on the bridge.

[0038] Step S1203: Calculate the rotation matrix and translation vector of the wide-angle camera based on the intrinsic parameter matrix of the wide-angle camera, the intrinsic parameter matrix of the telephoto camera, and the 3D pose of the wide-angle camera at each time step.

[0039] Step S1204: Calculate the Euclidean homography matrix of the wide-angle camera based on the rotation matrix and translation vector of the wide-angle camera.

[0040] Step S1205: Calculate the projection homography matrix of the telephoto camera based on the intrinsic parameter matrix of the telephoto camera and the Euclidean homography matrix of the wide-angle camera.

[0041] Step S1206: Perform UAV self-vibration elimination processing on the vibration video of the bridge cable based on the projection homography matrix of the telephoto camera to obtain the absolute vibration video of the bridge cable.

[0042] Further, in the above embodiment, step S1204 includes: The Euclidean homography matrix of a wide-angle camera can be calculated using the following formula:

[0043] in, H e The Euclidean homography matrix representing a wide-angle camera. The rotation matrix representing a wide-angle camera. The translation vector representing the wide-angle camera. n T The transpose matrix represents the unit normal vector of the bridge's fixed target plane in the first camera coordinate system. d This represents the distance from the wide-angle camera to the fixed target plane of the bridge.

[0044] Furthermore, in the above embodiment, step S1205 includes: The projection homography matrix of a telephoto camera is calculated using the following formula:

[0045] in, H p The projection homography matrix representing a telephoto camera, K t The intrinsic parameter matrix representing a telephoto camera, represent K t The inverse matrix.

[0046] In this embodiment of the invention, the wide-angle camera and the telephoto camera are first calibrated using Zhang's calibration method to obtain their respective intrinsic parameter matrices. Since the UAV is equipped with a dual-camera system, the wide-angle camera and the telephoto camera must be calibrated first before calibrating the dual-camera system. (Camera intrinsic parameter matrices) K The general form is as follows:

[0047] in, f x and f y Focal length is measured in pixels. c x and c y These are the coordinates of the principal point of the image.

[0048] For wide-angle cameras, since the focal length is a constant, the intrinsic parameter matrix of the wide-angle camera can be obtained through a single calibration. K w For telephoto cameras, since their focal length changes during measurement, calibration is required for multiple focal lengths. In actual measurements, the camera's focal length changes at fixed magnifications: 2x, 5x, 10x, 20x, 40x, and 80x zoom. Of these, only 5x, 10x, and 20x zoom magnifications are likely used in the measurements. Therefore, calibration is performed only for these three zoom magnifications, resulting in the intrinsic parameter matrix for the telephoto camera. K t Combining the intrinsic parameter matrix of a wide-angle cameraK w The intrinsic parameter matrix of the telephoto camera is K t Through joint calibration, the rotation matrix from wide-angle camera to telephoto camera can be obtained. R Translation vector T .

[0049] Then, the 3D pose (i.e., position and orientation) of the wide-angle camera at each time step is calculated using the vibration video of the fixed bridge target captured by the wide-angle camera. Finally, the rotation matrix of the wide-angle camera is calculated based on its 3D pose. Translation vector And thus, the Euclidean homography matrix of the UAV's self-vibration in the wide-angle camera is calculated. H e Its expression is:

[0050] in, H e The Euclidean homography matrix representing a wide-angle camera. The rotation matrix representing a wide-angle camera. The translation vector representing the wide-angle camera. n T The transpose matrix represents the unit normal vector of the bridge's fixed target plane in the first camera coordinate system. d This represents the distance from the wide-angle camera to the fixed target plane of the bridge.

[0051] Combined with the intrinsic parameter matrix of a telephoto camera K t Euclidean homography matrix of a wide-angle camera H e It can calculate the projection homography matrix of a telephoto camera. H p Its expression is:

[0052] in, H p The projection homography matrix representing a telephoto camera, K t The intrinsic parameter matrix representing a telephoto camera, represent K t The inverse matrix.

[0053] Finally, the projection homography matrix of the telephoto camera is utilized. H pThe vibration video of the bridge cable is processed to eliminate the interference of the drone's own vibration, so as to obtain the absolute vibration video of the bridge cable.

[0054] Optionally, in the above embodiments, step S140 includes: Step S1401: Extract pixel-level edge points of the bridge cables from the segmented absolute vibration video of the bridge cables using an edge detection algorithm.

[0055] Step S1402: Perform sub-pixel detection on each pixel-level edge point using a preset detection box, and calculate the sub-pixel coordinates of each pixel-level edge point using the local area effect to obtain the sub-pixel-level edge points of the bridge cable.

[0056] Step S1403: Use an edge tracking algorithm to track the sub-pixel level edge points of the bridge cable to obtain the full-field displacement of the bridge cable.

[0057] Furthermore, in the above embodiments, before step S1401, the method further includes: The absolute vibration video of the bridge cables after segmentation was smoothed using a Gaussian smoothing kernel.

[0058] In embodiments of the present invention, such as Figure 3 As shown, for the straight edge of a bridge cable, its edge equation can be expressed as: ,in, a and b The coefficients of the linear function are given. The straight edges of the bridge cables divide the image into two regions with different gray values.

[0059] Any pixel on the edge P The grayscale value can be represented as:

[0060] in, for P The grayscale value of a dot. A and B These represent the grayscale values ​​on both sides of the straight edge of the bridge cable. S A and S B These are two grayscale values. A and B At pixel P The area covered by the middle h The side length of each pixel is usually taken as... h =1, and Then we have:

[0061] To reduce the impact of background noise, a 3×3 area was used before edge detection began. A Gaussian smoothing kernel (GSK) is used to smooth video images of bridge cables. Gaussian smoothing blurs the image by attenuating high-frequency measurement noise and suppressing background details; this is achieved through image convolution. A size of 3×3 indicates that a 3×3 GSK is used to convolve 3×3 pixels, and the resulting gray values ​​are weighted and distributed to the center pixel. The expression for the 2D Gaussian function is:

[0062] in, is the standard deviation of the Gaussian function.

[0063] Determining the precise sub-pixel position of the straight edge of a bridge cable requires calculating the vertical distance from the pixel center to the edge (i.e., b ).like Figure 4 As shown, a 5×3 detection box is used to perform subpixel detection to determine the subpixel edge position of the straight line edge.

[0064] The sum of the grayscale values ​​in each column within the detection box can be expressed as:

[0065]

[0066]

[0067] in, L , M and R These are the sums of the grayscale values ​​in the left, middle, and right columns of the detection box, respectively. S L , S M and S R The area under the edges of each column can be calculated as follows:

[0068]

[0069]

[0070] Solving the above formula, we can obtain the coefficients. a and b for:

[0071]

[0072] Subpixel edge detection accuracy is affected by grayscale value. A and B The factors affecting edge detection accuracy are the average gray values ​​of the image background and the bridge cables, respectively. To reduce the impact of image noise on edge detection accuracy, the average gray values ​​of a 2x5 area (or even larger area) at the bottom (completely covered by the background) and top (completely covered by the cables) of the 5×3 detection box are used to calculate the edge detection accuracy. A and B . A and B It can be calculated as follows:

[0073]

[0074] Once the sub-pixel edge points of the bridge cable are obtained, the full-field displacement of the bridge cable is obtained by tracking all the sub-pixel edge points of the cable using an edge tracking algorithm.

[0075] Optionally, in the above embodiments, step S150 includes: Step S1501: Construct the Hankel matrix based on the vibration displacement.

[0076] Step S1502: Perform a projection transformation on the Hankel matrix to obtain the projection matrix.

[0077] Step S1503: Perform QR decomposition and singular value decomposition on the projection matrix, and calculate the eigenvalues ​​and eigenvectors using the least squares method.

[0078] Step S1504: Obtain the full-field vibration mode of the cable based on the eigenvalues ​​and eigenvectors.

[0079] In this embodiment of the invention, an oscillating system without a defined input can be represented using a state space through a data-driven random subspace identification method. A Hankel matrix is ​​constructed using the vibration displacements obtained in the preceding steps, and the Hankel matrix is ​​then divided into past output matrices. Y f and future output matrix Y p , matrix Y f Projection to matrix Y p Calculate the orthogonal projection matrix and to Perform QR decomposition.

[0080] According to the subspace system recognition theory, the projection matrix It can be decomposed into an observation matrix. and Kalman filter state matrix The product of the projection matrix. Perform Singular value decomposition (SVD), and then use the least squares method to calculate the eigenvalues. and eigenvectors Then, the frequency of the cable can be calculated. Damping ratio and mode This allows for the automatic acquisition of the full-field vibration modes of the bridge cables.

[0081] Another aspect of the present invention provides a bridge cable vibration measurement device based on UAV dual-vision fusion, and the device described below can be referred to in correspondence with the method described above.

[0082] Please see Figure 5 The device includes: The acquisition module 510 is used to acquire vibration videos of the fixed bridge target and the bridge cables, respectively, through a wide-angle camera and a telephoto camera connected to the UAV, wherein the wide-angle camera and the telephoto camera are set to be out of axis. The vibration cancellation module 520 is used to perform self-vibration cancellation processing on the vibration video of the bridge cable based on the vibration video of the fixed target of the bridge, using homography transformation, so as to obtain the absolute vibration video of the bridge cable. The segmentation processing module 530 is used to segment the absolute vibration video of the bridge cable using a trained neural network segmentation model to obtain the segmented absolute vibration video of the bridge cable. The displacement extraction module 540 is used to extract displacement from the segmented absolute vibration video of the bridge cable using a sub-pixel edge detection algorithm, so as to obtain the full-field displacement of the bridge cable. The mode shape acquisition module 550 is used to obtain the full-field mode shape of the bridge cable based on the full-field displacement of the bridge cable using a data-driven random subspace identification method.

[0083] As described above, the bridge cable vibration measurement device based on UAV dual-vision fusion provided in this embodiment of the invention acquires vibration videos of a fixed bridge target and vibration videos of the bridge cables, respectively, collected by a wide-angle camera and a telephoto camera connected to the UAV. The wide-angle camera and the telephoto camera are set off-axis. Based on the vibration video of the fixed bridge target, homography transformation is used to perform UAV self-vibration elimination processing on the vibration video of the bridge cables to obtain the absolute vibration video of the bridge cables. A trained neural network segmentation model is used to segment the absolute vibration video of the bridge cables to obtain the segmented absolute vibration video of the bridge cables. A sub-pixel edge detection algorithm is used to extract displacement from the segmented absolute vibration video of the bridge cables to obtain the full-field displacement of the bridge cables. Based on the full-field displacement of the bridge cables, a data-driven random subspace recognition method is used to obtain the full-field mode shape of the bridge cables. Using the solution of this application, the influence of the UAV's own vibration on the bridge vibration measurement results can be eliminated, thereby improving the accuracy and practicality of UAV-based vibration measurement technology.

[0084] Another aspect of the present invention provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the steps of the bridge cable vibration measurement method based on UAV dual-vision fusion as described above.

[0085] In another aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for measuring bridge cable vibration based on UAV dual-vision fusion.

[0086] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. It should also be noted that in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0087] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for measuring bridge cable vibration based on UAV dual-vision fusion, characterized in that, include: Vibration videos of a fixed bridge target and bridge cables are acquired by a wide-angle camera and a telephoto camera connected to a drone, respectively. The wide-angle camera and the telephoto camera are set to be non-axial. Based on the vibration video of the fixed target of the bridge, the vibration video of the bridge cable is processed by the UAV to eliminate its own vibration, so as to obtain the absolute vibration video of the bridge cable. The absolute vibration video of the bridge cable is segmented using a trained neural network segmentation model to obtain the segmented absolute vibration video of the bridge cable. The displacement is extracted from the absolute vibration video of the segmented bridge cable using a subpixel edge detection algorithm to obtain the full-field displacement of the bridge cable. Based on the full-field displacement of the bridge cables, the full-field vibration modes of the bridge cables are obtained using a data-driven random subspace identification method.

2. The method for measuring bridge cable vibration based on UAV dual-vision fusion according to claim 1, characterized in that, The step of performing self-vibration cancellation processing on the vibration video of the bridge cables using homography transformation based on the vibration video of the fixed target of the bridge to obtain the absolute vibration video of the bridge cables includes: Obtain the intrinsic parameter matrix of the wide-angle camera and the intrinsic parameter matrix of the telephoto camera; Based on the vibration video of the fixed target on the bridge, the 3D pose of the wide-angle camera at each time step is calculated; Based on the intrinsic parameter matrix of the wide-angle camera, the intrinsic parameter matrix of the telephoto camera, and the 3D pose of the wide-angle camera at each time step, calculate the rotation matrix and translation vector of the wide-angle camera. Calculate the Euclidean homography matrix of the wide-angle camera based on its rotation matrix and translation vector. The projection homography matrix of the telephoto camera is calculated based on the intrinsic parameter matrix of the telephoto camera and the Euclidean homography matrix of the wide-angle camera. The vibration video of the bridge cable is processed by the UAV to eliminate its own vibration based on the projection homography matrix of the telephoto camera, so as to obtain the absolute vibration video of the bridge cable.

3. The method for measuring bridge cable vibration based on UAV dual-vision fusion according to claim 2, characterized in that, The step of calculating the Euclidean homography matrix of the wide-angle camera based on its rotation matrix and translation vector includes: The Euclidean homography matrix of the wide-angle camera is calculated using the following formula: ； in, H e The Euclidean homography matrix representing the wide-angle camera. The rotation matrix representing the wide-angle camera, This represents the translation vector of the wide-angle camera. n T The transpose of the unit normal vector of the fixed target plane of the bridge in the first camera coordinate system is represented by the matrix. d This represents the distance from the wide-angle camera to the fixed target plane of the bridge.

4. The method for measuring bridge cable vibration based on UAV dual-vision fusion according to claim 3, characterized in that, The step of calculating the projection homography matrix of the telephoto camera based on the intrinsic parameter matrix of the telephoto camera and the Euclidean homography matrix of the wide-angle camera includes: The projection homography matrix of the telephoto camera is calculated using the following formula: ； in, H p The projection homography matrix represents the telephoto camera. K t This represents the intrinsic parameter matrix of the telephoto camera. represent K t The inverse matrix.

5. The method for measuring bridge cable vibration based on UAV dual-vision fusion according to claim 1, characterized in that, The process of extracting displacement from the segmented absolute vibration video of the bridge cables using a sub-pixel edge detection algorithm to obtain the full-field displacement of the bridge cables includes: Pixel-level edge points of the bridge cables are extracted from the segmented absolute vibration video of the bridge cables using an edge detection algorithm; Subpixel detection is performed on each pixel-level edge point using a preset detection box, and the subpixel coordinates of each pixel-level edge point are calculated using the local area effect to obtain the subpixel-level edge points of the bridge cable. An edge tracking algorithm is used to track the sub-pixel level edge points of the bridge cables to obtain the full-field displacement of the bridge cables.

6. The method for measuring bridge cable vibration based on UAV dual-vision fusion according to claim 5, characterized in that, Before extracting pixel-level edge points of the bridge cables from the segmented absolute vibration video of the bridge cables using an edge detection algorithm, the method further includes: The absolute vibration video of the segmented bridge cables was smoothed using a Gaussian smoothing kernel.

7. The method for measuring bridge cable vibration based on UAV dual-vision fusion according to claim 1, characterized in that, The step of obtaining the full-field vibration modes of the bridge cables using a data-driven random subspace identification method based on the full-field displacement of the bridge cables includes: Construct the Hankel matrix based on the full-field displacement; The Hankel matrix is ​​transformed by projection to obtain the projection matrix; The projection matrix is ​​subjected to QR decomposition and singular value decomposition, and the eigenvalues ​​and eigenvectors are calculated using the least squares method. Based on the eigenvalues ​​and eigenvectors, the full-field vibration modes of the bridge cables are obtained.

8. A bridge cable vibration measurement device based on UAV dual-vision fusion, characterized in that, include: The acquisition module is used to acquire vibration videos of a fixed bridge target and vibration videos of bridge cables, respectively, captured by a wide-angle camera and a telephoto camera connected to a drone, wherein the wide-angle camera and the telephoto camera are set to be non-axial. The vibration cancellation module is used to perform self-vibration cancellation processing on the vibration video of the bridge cables based on the vibration video of the fixed target of the bridge, using homography transformation, so as to obtain the absolute vibration video of the bridge cables. The segmentation processing module is used to segment the absolute vibration video of the bridge cable using a trained neural network segmentation model to obtain the segmented absolute vibration video of the bridge cable. The displacement extraction module is used to extract displacement from the absolute vibration video of the segmented bridge cable using a subpixel edge detection algorithm, so as to obtain the full-field displacement of the bridge cable. The mode shape acquisition module is used to obtain the full-field mode shape of the bridge cable based on the full-field displacement of the bridge cable using a data-driven random subspace identification method.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the bridge cable vibration measurement method based on UAV dual-vision fusion as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the bridge cable vibration measurement method based on UAV dual-vision fusion as described in any one of claims 1 to 7.

Images