A target-free structural vibration signal acquisition method based on a UAV platform

Abstract

This invention discloses a targetless structural vibration signal acquisition method based on an unmanned aerial vehicle (UAV) platform, relating to the field of infrastructure structural health monitoring. The method includes the following steps: S1: A UAV is brought close to the infrastructure structure to acquire and preprocess video data, obtaining a video image v1 of the vibration process and a video image v2 of irregular motion; S2: Based on video image v2, the UAV motion is compensated to eliminate large displacement noise of integer pixels, resulting in a processed video image v3; S3: Based on video image v3, a video image v4 without the structure under test is obtained, and the UAV motion is compensated to eliminate small displacement noise of sub-pixels, resulting in a precisely motion-compensated video image v5; S4: Based on video image v5, optical flow estimation and post-processing are performed. This method achieves high-precision acquisition of local vibration information, overcomes the long-distance resolution limitations of fixed cameras, and enables rapid processing of target vibration images and output of vibration information.

Description

Technical Field

[0001] This invention relates to the field of infrastructure structural health monitoring, and in particular to a targetless structural vibration signal acquisition method based on an unmanned aerial vehicle (UAV) platform. Background Technology

[0002] Structural vibration analysis is one of the important methods for structural health monitoring (SHM) and equipment fault diagnosis. Traditional vibration signal measurement methods include contact accelerometers and non-contact optical sensors. Accelerometers need to be attached to the structure to achieve vibration measurement, which is relatively expensive, cumbersome to operate, and prone to quality problems due to circuit faults. The most widely used non-contact optical sensor is the Doppler laser vibrometer, which requires complex interferometric optical path auxiliary devices, is too expensive, and has a small measurement range, making it difficult to achieve large-scale, multi-point measurement of infrastructure structures.

[0003] In recent years, with the rise of computer technology, computer vision-based structural vibration measurement methods have attracted increasing attention from researchers in the field of structural health monitoring. Vision-based methods can achieve structural vibration measurement without additional loads, over a wide range, and at low cost.

[0004] Among existing patented technologies, for example, CN118706244A proposes a real-time bridge vibration monitoring technology and method based on visual enhancement. This method uses a fixed camera to capture vibration images of the bridge structure in real time, selects specific areas for image cropping, and performs lossless and lossy data compression processing. It then uses template matching combined with LK subpixel optical flow technology to extract vibration signals from the lossless compressed images and determines the fundamental frequency of the bridge through frequency domain analysis. The lossy compressed images are then processed for video synthesis, and multi-layer filtering technology is used to visualize and amplify the vibration of the bridge structure, effectively improving the accuracy of vibration signal extraction and achieving visualization and amplification of vibration through multi-layer filtering.

[0005] However, computer vision methods based on fixed cameras have certain limitations in application scenarios. In practical engineering applications, if the test area is far from the camera and exceeds the camera's resolution, it is difficult to extract high-precision vibration signals. Using mobile acquisition platforms such as drones for close-up shooting can effectively solve the shooting distance problem in the acquisition of signals from large real structures. However, drones will shake when hovering, causing camera movement, which will bring huge errors or mistakes to the estimation of structural vibration signals.

[0006] Therefore, a targetless structural vibration signal acquisition method based on an unmanned aerial vehicle (UAV) platform is provided to solve the above problems. Summary of the Invention

[0007] The purpose of this invention is to provide a targetless structural vibration signal acquisition method based on an unmanned aerial vehicle (UAV) platform. This method uses a UAV camera platform to perform non-contact measurement of vibration signals of infrastructure structures, effectively solving the scene limitation problem in existing fixed camera vibration signal measurement methods.

[0008] To achieve the above objectives, this invention provides a targetless structural vibration signal acquisition method based on an unmanned aerial vehicle (UAV) platform, comprising the following steps:

[0009] S1: By using drones to approach infrastructure structures to collect and preprocess video data, video images v1 of the vibration process and v2 of irregular motion are obtained;

[0010] S2: Compensate for the drone's motion based on video image v2, eliminate large displacement noise of whole pixels, and obtain the processed video image v3;

[0011] S3: Based on video image v3, obtain video image v4 without the structure under test. Based on video image v4, compensate for the motion of the UAV, eliminate subpixel small displacement noise, and obtain video image v5 after accurate motion compensation.

[0012] S4: Optical flow estimation and post-processing based on video image v5.

[0013] Preferably, step S1 specifically includes the following steps:

[0014] S11: Obtain video images v1 of the vibration process by taking video of the area under test using a drone;

[0015] S12: Obtain the static background of video image v1. When the drone hovers in the static background, the camera produces irregular motion. Obtain the video image v2 with irregular motion corresponding to the static background.

[0016] S13: Input the video image v2 as a time series frame by frame into the optical flow neural network FlowFormer.

[0017] Preferably, step S2 specifically includes the following steps:

[0018] S21: Train the optical flow neural network FlowFormer using the public dataset FlyingChairs, and use the optical flow neural network FlowFormer to calculate the optical flow of the image between frames;

[0019] S22: Calculate the target pixel displacement data using optical flow data;

[0020] S23: Based on integer pixel displacement data, a heuristic algorithm is used to perform heuristic image translation processing on video image v1 frame by frame, limiting the noise caused by camera motion to within a unit pixel, and generating the processed video image v3.

[0021] Preferably, in step S21, the input signal of the optical flow neural network FlowFormer is an image pair consisting of a video reference frame and the current frame, and the output signal of the optical flow neural network FlowFormer is the full-field optical flow data corresponding to the image pair.

[0022] Preferably, step S21 specifically includes the following steps:

[0023] Step 1: Enter H I ×W I The H×W×D image is extracted using a 3RGB image and a backbone visual network. f The feature map is used to extract the source image from the previous frame and the target image from the current frame. The dot product similarity of all pixels in the source image and the target image is calculated to construct a 4D cost volume of H×W×H×W, where (H,W)=(H I / 8,W I / 8);

[0024] Step 2: Identify the corresponding position of the source pixel in the target image by encoding the source-view visual similarity in the 4D cost volume, and eliminate the repetitive patterns and indistinguishable regions in the two frames of images by cost encoder;

[0025] Step 3: Using a cost decoder, predict the optical flow residual Δf(x) recursively. The optical flow residual Δf(x) is expressed as:

[0026] Δf(x)=ConvGRU(Concat(c x ,q x ,t x ,f(x)))

[0027] Among them, t x The contextual features of the source image are represented, and the optical flow prediction is optimized step by step using an iterative method. During the iteration process, the optical flow residual Δf(x) is regressed through a gated recurrent unit ConvGRU.

[0028] Preferably, step 2 specifically includes the following steps:

[0029] Step 1: Perform strided convolution on the cost map M of each source pixel x. x Block processing is performed, cost graph M x Represented as:

[0030]

[0031] Multiple cost blocks q are obtained x Embedded cost graph M x Multiple cost blocks q are activated by the ReLU activation function. x Embedded cost graph M x Convert to feature map F x Feature map F x Represented as:

[0032]

[0033] Among them, feature map F x Cost block features F x With the 8×8 cost block q in the cost graph x One-to-one correspondence;

[0034] Step 2: Randomly initialize the latent codeword C using K latent codewords. For each cost block feature F x Summarize the data and query the features F of each cost block using a dot product attention mechanism. x Through the latent representation T x The cost graph is summarized as K latent vectors of dimension D. The feature F of each cost block is determined using the COTR method. x The embedding location is used to obtain the 4D cost volume T, which is expressed as:

[0035]

[0036] The latent codeword C is updated through backpropagation and shared among different source pixels x.

[0037] Step 3: The 4D cost volume T is grouped using an alternating Transformer layer (AGT). Based on the grouping results, a corresponding number of cost queries Q are generated using a feedforward neural network (FFN). x Cost Inquiry Q x Represented as:

[0038] Q x =FFN(FFN(q) x )+PE(p))

[0039] Where, q x Indicates from cost graph M x Extract the 9×9 cost block centered at position p;

[0040] Step 4: Predict the optical flow of the source pixels using cost query, and calculate the position p of each source pixel x in the target image. The position p is represented as:

[0041] p = x + f(x)

[0042] Where f(x) represents the current optical flow estimate;

[0043] Step 5: Generate the cost memory information c of the aggregated source pixel x through a cross-attention mechanism. x Cost of remembering information c x Represented as:

[0044] c x =Attention(Q) x ,K x V x ).

[0045] Preferably, in step two, the potential representation T x Represented as:

[0046] K x =Conv 1×1 (Concat(F x ,PE))

[0047] V x =Conv 1×1 (Concat(F x ,PE))

[0048] T x =Attention(C,K) x V x )

[0049] Among them, key K x Sum V x All represent cost block features F x The projection, key K x Sum V x They are represented as follows:

[0050] K x =FFN(T x )

[0051] V x =FFN(T x ),

[0052] The cost block feature F x Projection as key K x Sum V x Previously, the cost block feature F x The positional embedding sequence PE is concatenated with the positional embedding sequence PE, which is represented as:

[0053]

[0054] Among them, D p Indicates the encoding length.

[0055] Preferably, in step three, the AGT layer specifically includes the following two grouping methods:

[0056] Method 1: Represent the latent value of each source pixel x Perform a set and set it as a group, then perform a set of all potential representations within the group. Spatial separation self-attention computation is performed to obtain the updated latent representation T. x The updated potential representation T x Represented as:

[0057]

[0058] Among them, T x (i) represents the i-th latent representation of the source pixel x, and the latent representation T calculated by the feedforward network FFN after self-attention. x The process involves reorganizing the 4D cost volume T to form the cost map M. x internal self-attention;

[0059] Method 2: Based on latent representation T x All 4D cost volumes T are grouped according to the given criteria, with each group containing H×W cost labels T. Spatial separation self-attention computation is then performed on all cost labels T within each group to obtain the updated cost labels T. i The updated cost marker T i Represented as:

[0060] T i =FFN(SS-SelfAttention(T) i )), i = 1, 2, ..., K.

[0061] Preferably, step S3 specifically includes the following steps:

[0062] S31: Based on video image v3, obtain the background without the structure to be tested, and set the reference area video image without the structure to be tested as video image v4. The coordinate positions of video image v4 and video image v2 are the same in video image v1.

[0063] S32: Combining the variational pyramid optical flow compensation algorithm KLT, feature points in video image v4 are extracted frame by frame, optical flow tracing is performed on the feature points, and the sub-pixel transformation relationship between different frames caused by camera motion is represented by the affine transformation matrix.

[0064] S33: Perform affine transformation on video image v4 frame by frame using the affine transformation matrix to eliminate subpixel small displacement noise in video image v3 and obtain video image v5 after accurate motion compensation.

[0065] Preferably, step S4 specifically includes the following steps:

[0066] S41: Based on video image v5, the region to be tested is set as the ROI region, and the local phase information of the image in the ROI region is extracted by a complex manipulating pyramid filter;

[0067] S42: Track the ROI region using the optical flow compensation algorithm KLT, calculate the sub-pixel level displacement time history data of the feature points in the region under test, multiply the pixel displacement time history data by the scaling factor, and convert the pixel displacement time history data into real displacement.

[0068] S43: Based on the real displacement data of multiple measuring points, the frequency domain power spectrum (PSD) is obtained by fast Fourier transform (FFT). Modal analysis is then performed on the PSD data using frequency decomposition (FDD) to obtain the modal information of the structure under test.

[0069] Therefore, the present invention employs the above-mentioned targetless structural vibration signal acquisition method based on an unmanned aerial vehicle platform, which has the following beneficial effects:

[0070] (1) This invention does not require an additional target and can extract vibration information from multiple measuring points from structural vibration videos captured by drones, and further obtain the overall modal information of the structure through the vibration information;

[0071] (2) This invention effectively reduces costs, achieves high-precision imaging of local vibration information, overcomes the long-distance resolution limitation of fixed cameras, and can quickly process target vibration images and output vibration information.

[0072] The method of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0073] Figure 1 This is a flowchart of a targetless structural vibration signal acquisition method based on an unmanned aerial vehicle (UAV) platform according to the present invention.

[0074] Figure 2 This is a diagram of the FlowFormer optical flow neural network architecture of the present invention. Detailed Implementation

[0075] The method of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0076] Unless otherwise defined, the methodological or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0077] The terms "comprising" or "including" as used in this invention mean that the element preceding the term encompasses the element listed after the term, and do not exclude the possibility of encompassing other elements. Terms such as "inner," "outer," "upper," and "lower" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In this invention, unless otherwise explicitly specified and limited, the term "attached" and similar terms should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral part; it can refer to a direct connection or an indirect connection through an intermediate medium; it can refer to the internal communication of two elements or the interaction relationship between two elements. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0078] Example

[0079] like Figure 1 and Figure 2 As shown, this invention provides a targetless structural vibration signal acquisition method based on an unmanned aerial vehicle (UAV) platform, comprising the following steps:

[0080] S1: By using drones to approach infrastructure structures to collect and preprocess video data, video images v1 of the vibration process and v2 of irregular motion are obtained;

[0081] Step S1 specifically includes the following steps:

[0082] S11: Obtain video images v1 of the vibration process by taking video of the area under test using a drone;

[0083] S12: Obtain the static background of video image v1. When the drone hovers in the static background, the camera produces irregular motion. Obtain the video image v2 with irregular motion corresponding to the static background.

[0084] S13: Input the video image v2 as a time series frame by frame into the optical flow neural network FlowFormer.

[0085] S2: Compensate for the drone's motion based on video image v2, eliminate large displacement noise of whole pixels, and obtain the processed video image v3;

[0086] Step S2 specifically includes the following steps:

[0087] S21: Train the optical flow neural network FlowFormer using the public dataset FlyingChairs, and use the optical flow neural network FlowFormer to calculate the optical flow of the image between frames;

[0088] In step S21, the input signal of the optical flow neural network FlowFormer is an image pair consisting of a video reference frame and the current frame, and the output signal of the optical flow neural network FlowFormer is the full-field optical flow data corresponding to the image pair.

[0089] Step S21 specifically includes the following steps:

[0090] Step 1: Enter H I ×W I The H×W×D image is extracted using a 3RGB image and a backbone visual network. f The feature map is used to extract the source image from the previous frame and the target image from the current frame. The dot product similarity of all pixels in the source image and the target image is calculated to construct a 4D cost volume of H×W×H×W, where (H,W)=(H I / 8,W I / 8);

[0091] Step 2: Identify the corresponding position of the source pixel in the target image by encoding the source-view visual similarity in the 4D cost volume, and eliminate the repetitive patterns and indistinguishable regions in the two frames of images by cost encoder;

[0092] Step 2 specifically includes the following steps:

[0093] Step 1: Perform strided convolution on the cost map M of each source pixel x. x Block processing is performed, cost graph M x Represented as:

[0094]

[0095] Multiple cost blocks q are obtained x Embedded cost graph M x Multiple cost blocks q are activated by the ReLU activation function. x Embedded cost graph M x Convert to feature map F x Feature map F x Represented as:

[0096]

[0097] Among them, feature map F x Cost block features F x With the 8×8 cost block q in the cost graphx One-to-one correspondence;

[0098] Step 2: Randomly initialize the latent codeword C using K latent codewords. For each cost block feature F x Summarize the data and query the features F of each cost block using a dot product attention mechanism. x Through the latent representation T x The cost graph is summarized as K latent vectors of dimension D. The feature F of each cost block is determined using the COTR method. x The embedding location is used to obtain the 4D cost volume T, which is expressed as:

[0099]

[0100] The above steps transform the original 4D cost volume into a compact potential 4D cost volume T.

[0101] The latent codeword C is updated through backpropagation and shared among different source pixels x.

[0102] In step two, the latent representation T x Represented as:

[0103] K x =Conv 1×1 (Concat(F x ,PE))

[0104] V x =Conv 1×1 (Concat(F x ,PE))

[0105] T x =Attention(C,K) x V x )

[0106] Among them, key K x Sum V x All represent cost block features F x The projection, key K x Sum V x They are represented as follows:

[0107] K x =FFN(T x )

[0108] V x =FFN(T x ),

[0109] The cost block feature Fx Projection as key K x Sum V x Previously, the cost block feature F x The positional embedding sequence PE is concatenated with the positional embedding sequence PE, which is represented as:

[0110]

[0111] Among them, D p Indicates the encoding length.

[0112] Step 3: To reduce the geometrical increase in complexity and computational cost caused by the increasing number of labeled vectors, the 4D cost volume T is grouped using alternating Transformer layers (AGT). Based on the grouping results, a corresponding number of cost queries Q are generated using a feedforward neural network (FFN). x Cost Inquiry Q x Represented as:

[0113] Q x =FFN(FFN(q) x )+PE(p))

[0114] Where, q x Indicates from cost graph M x Extract the 9×9 cost block centered at position p;

[0115] In step three, the AGT layer specifically includes the following two grouping methods:

[0116] Method 1: Represent the latent value of each source pixel x Perform a set and set it as a group, then perform a set of all potential representations within the group. Spatial separation self-attention computation is performed to obtain the updated latent representation T. x The updated potential representation T x Represented as:

[0117]

[0118] Among them, T x (i) represents the i-th latent representation of the source pixel x, and the latent representation T calculated by the feedforward network FFN after self-attention. x The process involves reorganizing the 4D cost volume T to form the cost map M. x internal self-attention;

[0119] Method 2: Based on latent representation T xAll 4D cost volumes T are grouped according to the given criteria, with each group containing H×W cost labels T. Spatial separation self-attention computation is then performed on all cost labels T within each group to obtain the updated cost labels T. i The updated cost marker T i Represented as:

[0120] T i =FFN(SS-SelfAttention(T) i )), i = 1, 2, ..., K.

[0121] By alternating between these two methods, AGT can effectively exchange information between source pixels and latent representations, ultimately transforming the 4D cost volume into a more compact cost memory for decoding optical flow estimation.

[0122] Step 4: Predict the optical flow of the source pixels using cost query, and calculate the position p of each source pixel x in the target image. The position p is represented as:

[0123] p = x + f(x)

[0124] Where f(x) represents the current optical flow estimate;

[0125] Step 5: Generate the cost memory information c of the aggregated source pixel x through a cross-attention mechanism. x Cost of remembering information c x Represented as:

[0126] c x =Attention(Q) x ,K x V x ).

[0127] Step 3: Using a cost decoder, predict the optical flow residual Δf(x) recursively. The optical flow residual Δf(x) is expressed as:

[0128] Δf(x)=ConvGRU(Concat(c x ,q x ,t x ,f(x)))

[0129] Among them, t x The contextual features of the source image are represented, and the optical flow prediction is optimized step by step using an iterative method. During the iteration process, the optical flow residual Δf(x) is regressed through a gated recurrent unit ConvGRU.

[0130] S22: Calculate the target pixel displacement data using optical flow data;

[0131] S23: Based on integer pixel displacement data, a heuristic algorithm is used to perform heuristic image translation processing on video image v1 frame by frame, limiting the noise caused by camera motion to within a unit pixel, and generating the processed video image v3.

[0132] S3: Based on video image v3, obtain video image v4 without the structure under test. Based on video image v4, compensate for the motion of the UAV, eliminate subpixel small displacement noise, and obtain video image v5 after accurate motion compensation.

[0133] Step S3 specifically includes the following steps:

[0134] S31: Based on video image v3, obtain the background without the structure to be tested, and set the reference area video image without the structure to be tested as video image v4. The coordinate positions of video image v4 and video image v2 are the same in video image v1.

[0135] S32: During video stabilization, the user first needs to manually select a stationary reference region in the area to be tested as the benchmark for subsequent optical flow calculation. This operation is achieved through mouse interaction. The user selects a stationary region in the first frame and uses the OpenCV library callback function cv2.setMouseCallback to register mouse events to capture the user's input. This function listens for mouse events and calls the select_roi callback function to determine the region based on mouse clicks and drags. After the user has made the selection, the coordinates of the selected region are extracted for subsequent processing.

[0136] Combining the variational pyramid optical flow compensation algorithm KLT, feature points are extracted from video image v4 frame by frame. The OpenCV library function cv2.goodFeaturesToTrack() is used to detect easily trackable corner points in the image. Through detection, 100 feature points are obtained for tracking. These feature points will be tracked in subsequent frames using the optical flow method. The processing of each frame is based on the optical flow method. The motion of the camera or object is estimated by calculating the amount of movement of the feature points between the current frame and the previous frame. The optical flow method is implemented by the OpenCV library function cv2.calcOpticalFlowPyrLK().

[0137] To eliminate subpixel noise from camera shake, the translation between frames is estimated by tracking the movement of these feature points. For each frame, the average displacement of the feature points is calculated to obtain the translation vectors in the horizontal and vertical directions. The displacement of the feature points is obtained by calculating the difference between the coordinates of the feature points in the current frame and the coordinates of the feature points in the previous frame, and then the average value is taken to obtain the overall translation.

[0138] After obtaining the inter-frame translation, the sub-pixel transformation relationship between different frames caused by camera motion is represented by an affine transformation matrix;

[0139] S33: Perform an affine transformation on video image v4 frame by frame using an affine transformation matrix, and adjust the current frame back to its original position using a translation transformation matrix to compensate for subpixel jitter.

[0140] The OpenCV library function cv2.warpAffine() is used to perform image translation matrix transformation. After processing the current frame, the current frame is used as the previous frame of the next frame, and the above optical flow calculation and correction process is repeated. In order to maintain the consistency of feature points, the tracked feature points are updated for the next iteration.

[0141] This eliminates subpixel small displacement noise in video image v3, resulting in video image v5 with accurate motion compensation.

[0142] First, a heuristic algorithm is used to perform integer pixel correction on the video image, which limits the camera's movement to within one pixel. This effectively eliminates vibration noise caused by large movements when the drone is hovering, creating the necessary conditions for the next step of using the optical flow frame-by-frame motion compensation algorithm. This effectively solves the problem of optical flow tracking algorithm failure caused by drone movement.

[0143] S4: Optical flow estimation and post-processing based on video image v5, implemented using Python and OpenCV modules.

[0144] Step S4 specifically includes the following steps:

[0145] S41: Based on video image v5, the region to be tested is set as the ROI region, and the local phase information of the image in the ROI region is extracted by a complex manipulating pyramid filter;

[0146] S42: Track the ROI region using the optical flow compensation algorithm KLT, calculate the sub-pixel level displacement time history data of the feature points in the region under test, multiply the pixel displacement time history data by the scaling factor, and convert the pixel displacement time history data into real displacement.

[0147] S43: Based on the real displacement data of multiple measuring points, the frequency domain power spectrum (PSD) is obtained by fast Fourier transform (FFT). Modal analysis is then performed on the PSD data using frequency decomposition (FDD) to obtain the modal information of the structure under test.

[0148] Therefore, the present invention adopts the above-mentioned targetless structural vibration signal acquisition method based on UAV platform, and performs integer-pixel large displacement denoising and sub-pixel small displacement denoising based on computer vision method, and performs displacement tracking and modal analysis, realizing high-precision shooting of local vibration information, overcoming the long-distance resolution limitation of fixed camera, and can quickly process target vibration images and output vibration information.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the method of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the method of the present invention, and these modifications or equivalent substitutions should not cause the modified method to deviate from the spirit and scope of the method of the present invention.

Claims

1. A method for unmanned aerial vehicle platform-based non-targeted structural vibration signal acquisition, characterized in that, Includes the following steps: S1: By using drones to approach infrastructure structures to collect and preprocess video data, video images v1 of the vibration process and v2 of irregular motion are obtained; S11: Obtain video images v1 of the vibration process by taking video of the area under test using a drone; S12: Obtain the static background of video image v1. When the drone hovers in the static background, the camera produces irregular motion. Obtain the video image v2 with irregular motion corresponding to the static background. S13: Input the video image v2 as a time series frame by frame into the optical flow neural network FlowFormer; S2: Compensate for the drone's motion based on video image v2, eliminate large displacement noise of whole pixels, and obtain the processed video image v3; S21: Train the optical flow neural network FlowFormer using the public dataset FlyingChairs, and use the optical flow neural network FlowFormer to calculate the optical flow of the image between frames; S22: Calculate the target pixel displacement data using optical flow data; S23: Based on integer pixel displacement data, a heuristic algorithm is used to perform heuristic image translation processing on video image v1 frame by frame to limit the noise caused by camera motion to within a unit pixel, and generate the processed video image v3. S3: Based on video image v3, obtain video image v4 without the structure under test. Based on video image v4, compensate for the motion of the UAV, eliminate subpixel small displacement noise, and obtain video image v5 after accurate motion compensation. S31: Based on video image v3, obtain the background without the structure to be tested, and set the reference area video image without the structure to be tested as video image v4. The coordinate positions of video image v4 and video image v2 are the same in video image v1. S32: Combining the variational pyramid optical flow compensation algorithm KLT, feature points in video image v4 are extracted frame by frame, optical flow tracing is performed on the feature points, and the sub-pixel transformation relationship between different frames caused by camera motion is represented by the affine transformation matrix. S33: Perform affine transformation on video image v4 frame by frame using affine transformation matrix to eliminate subpixel small displacement noise in video image v3 and obtain video image v5 after accurate motion compensation. S4: Optical flow estimation and post-processing based on video image v5; S41: Based on video image v5, the region to be tested is set as the ROI region, and the local phase information of the image in the ROI region is extracted by a complex manipulating pyramid filter; S42: Track the ROI region using the optical flow compensation algorithm KLT, calculate the sub-pixel level displacement time history data of the feature points in the region under test, multiply the pixel displacement time history data by the scaling factor, and convert the pixel displacement time history data into real displacement. S43: Based on the real displacement data of multiple measuring points, the frequency domain power spectrum (PSD) is obtained by fast Fourier transform (FFT). Modal analysis is then performed on the PSD data using frequency decomposition (FDD) to obtain the modal information of the structure under test.

2. The method for acquiring targetless structural vibration signals based on an unmanned aerial vehicle (UAV) platform according to claim 1, characterized in that, In step S21, the input signal of the optical flow neural network FlowFormer is an image pair consisting of a video reference frame and the current frame, and the output signal of the optical flow neural network FlowFormer is the full-field optical flow data corresponding to the image pair.

3. The method for acquiring targetless structural vibration signals based on an unmanned aerial vehicle (UAV) platform according to claim 2, characterized in that, Step S21 specifically includes the following steps: Step 1: Input Images are extracted using the backbone visual network. The feature map is used to extract the source image from the previous frame and the target image from the current frame. The dot product similarity of all pixels in the source and target images is calculated to construct... The 4D cost volume, of which, ; Step 2: Identify the corresponding position of the source pixel in the target image by encoding the source-view visual similarity in the 4D cost volume, and eliminate the repetitive patterns and indistinguishable regions in the two frames of images by cost encoder; Step 3: Predict the optical flow residual using a recursive approach via a cost decoder. optical flow residual Represented as: in, The optical flow prediction is progressively optimized using an iterative approach, representing the contextual features of the source image. During the iteration process, the optical flow residual is regressed through a gated recurrent unit (ConvGRU). ; Indicates from the cost diagram Extract by position Centered The cost block; This represents the current optical flow estimate; Represents the source pixel x The cost of remembering information.

4. The method for acquiring targetless structural vibration signals based on an unmanned aerial vehicle (UAV) platform according to claim 3, characterized in that, Step 2 specifically includes the following steps: Step 1: Perform strided convolution on each source pixel. x Cost diagram Block processing, cost graph Represented as: ， Obtain multiple cost blocks Cost graph of embedding Multiple cost blocks are processed using the ReLU activation function. Cost graph of embedding Convert to feature map Feature map Represented as: Among them, feature map Cost block features With cost diagram Cost block One-to-one correspondence; Indicates the encoding length; Step 2: Analyze potential code words Perform random initialization, through A potential code word For each cost block feature Summarize the data and query the features of each cost block using a dot product attention mechanism. Through latent representation The cost diagram is summarized as follows: Each dimension is latent vectors The COTR method is used to determine the features of each cost block. The embedding location is used to obtain the 4D cost volume. 4D cost volume Represented as: ， Update the potential codeword through backpropagation. and at different source pixels x Shared potential code words ; Step 3: Calculate the 4D cost volume using alternating grouped Transformer layers (AGT). Grouping is performed, and based on the grouping results, a corresponding number of cost queries are generated using a feedforward neural network (FFN). Cost Inquiry Represented as: in, Indicates from the cost diagram Extract by position Centered The cost block; Step 4: Predict the optical flow of the source pixels using cost query, and calculate the optical flow of each source pixel. The corresponding position in the target image ,Location Represented as: in, This represents the current optical flow estimate; Step 5: Generate aggregated source pixels using a cross-attention mechanism x The cost of memory information Cost of remembering information Represented as: 。 5. The method for acquiring targetless structural vibration signals based on an unmanned aerial vehicle (UAV) platform according to claim 4, characterized in that, In step two, the latent representation Represented as: Among them, key Sum All represent cost block features The projection, key Sum They are represented as follows: ， Cost block features Projection as key Sum Previously, for cost block features and position embedding sequence Perform splicing processing, position embedding sequence Represented as: in, Indicates the encoding length.

6. The method for acquiring targetless structural vibration signals based on an unmanned aerial vehicle (UAV) platform according to claim 5, characterized in that, In step three, the AGT layer specifically includes the following two grouping methods: Method 1: Convert each source pixel x Potential representation Perform a set and set it as a group, then perform a set of all potential representations within the group. Spatial separation self-attention computation is performed to obtain the updated latent representation. The updated potential representation Represented as: in, Represents the source pixel x The A latent representation, calculated by a feedforward network FFN after self-attention. Processing and reorganizing into 4D cost volume To form a cost map internal self-attention; Method 2: Based on latent representation All 4D cost volumes Based on the grouping, each group contains Cost marker Mark all costs within the group. Perform spatial separation self-attention computation to obtain the updated cost label. Updated cost marker Represented as: 。

Images