A structural vibration monitoring method and system based on unmanned aerial vehicles (UAVs)

Abstract

This application provides a method and system for structural vibration monitoring based on unmanned aerial vehicles (UAVs), relating to the technical field of structural health monitoring technology. The method includes: acquiring vibration information of a target cable; performing information enhancement and segmentation processing on the target cable vibration information to obtain processed target cable vibration information; extracting displacement from the target cable vibration information using a preset first algorithm to obtain the displacement time history of the target cable; performing UAV self-vibration elimination processing on the displacement time history of the target cable using a preset second algorithm to obtain the absolute displacement time history of the target cable; and estimating cable force on the absolute displacement time history of the target cable using a preset first model to obtain cable force information of the target cable. This invention solves the problem of self-vibration when using UAVs for structural vibration measurement, thereby achieving accurate identification of structural displacement.

Description

Technical Field

[0001] The present invention relates to the field of structural health monitoring technology, and more specifically, to a method and system for structural vibration monitoring based on unmanned aerial vehicles (UAVs). Background Technology

[0002] With the development of my country's infrastructure, civil infrastructure such as bridges, buildings, and dams are playing an increasingly important role in people's daily activities. Among them, the number of highway bridges has grown rapidly, reaching a total of 1.0332 million in my country, while the number of dangerous bridges has reached over 100,000. As the main load-bearing component of cable-stayed bridges, cables are easily damaged by traffic loads and external environmental factors, leading to a decrease in their load-bearing capacity. Therefore, monitoring the cable tension is essential.

[0003] Traditional cable force testing methods rely on contact sensors, which require manual placement of the sensors, resulting in high testing costs and low efficiency. Furthermore, the high traffic volume on bridges compromises safety. On the other hand, using drones for inspection can lead to insufficient testing accuracy due to the significant vibrations inherent in the drone's structure.

[0004] Therefore, a new solution is needed to address the above problems. Summary of the Invention

[0005] This invention provides a method and system for monitoring structural vibration based on unmanned aerial vehicles (UAVs), which at least solves the problem of self-vibration when using UAVs to measure structural vibration in related technologies.

[0006] According to an embodiment of the present invention, a method for monitoring structural vibration based on an unmanned aerial vehicle (UAV) is provided, comprising:

[0007] The target cable vibration information is obtained by preprocessing the initial target cable vibration information.

[0008] The target cable vibration information is subjected to information enhancement and segmentation processing to obtain the processed target cable vibration information;

[0009] Displacement is extracted from the vibration information of the target cable using a preset first algorithm to obtain the displacement time history of the target cable;

[0010] The displacement time history of the target cable is processed by the UAV's self-vibration elimination through a preset second algorithm to obtain the absolute displacement time history of the target cable.

[0011] The cable force of the target cable is estimated by using a preset first model to estimate the absolute displacement time history of the target cable.

[0012] In one exemplary embodiment,

[0013] The preprocessing of the initial target cable vibration information includes:

[0014] Based on the initial target cable information, an information sequence is determined;

[0015] Noise detection is performed on the information sequence to obtain the position information of the target cable, wherein the position information includes first coordinate information and first coordinate system information;

[0016] Based on the information sequence, the first coordinate information, and the first coordinate system information, a coordinate system transformation process is performed to obtain the position matrix information;

[0017] Based on the position matrix information, the preset target matrix is ​​solved to obtain the target cable vibration information.

[0018] In one exemplary embodiment,

[0019] The step of extracting displacement from the vibration information of the target cable using a preset first algorithm to obtain the displacement time history of the target cable includes:

[0020] Based on the vibration information of the target cable, the displacement coordinate information of the target cable is determined, wherein the displacement coordinate information includes second coordinate information and gradient vector information;

[0021] Based on the second coordinate information, the sub-coordinate information and sub-gradient vector information of the target cable are determined;

[0022] Gradient correlation is calculated on the gradient vector information and the sub-gradient information. If the correlation calculation result satisfies the correlation condition, the second coordinate information and the sub-coordinate information are processed into element point set to obtain an initial coordinate point set.

[0023] Based on the initial set of coordinate points, coordinate transformation calculations are performed to obtain the initial coordinate transformation matrix;

[0024] Based on the initial coordinate point set, an update iteration calculation is performed on the initial coordinate transformation matrix to obtain a target transformation matrix that satisfies the incremental condition and the corresponding target point set; wherein, the incremental condition includes that the coordinate increment in the sub-coordinate information is less than the target element value; the update iteration calculation includes: minimizing the initial coordinate transformation matrix to obtain a first transformation matrix; performing point set transformation processing on the initial coordinate point set based on the first transformation matrix to obtain a first point set, and updating the target point set with the sub-coordinate information and sub-gradient vector information that are closest to the coordinate points in the initial coordinate point set and the first point set;

[0025] The displacement time history of the target cable is determined based on the target transformation matrix and the target point set.

[0026] In an exemplary embodiment, the step of performing self-vibration elimination processing on the displacement time history of the target cable using a preset second algorithm to obtain the absolute displacement time history of the target cable includes:

[0027] The displacement time history of the target cable is subjected to noise estimation processing using the second algorithm to obtain a first estimation result;

[0028] Based on the first coefficient, wavelet threshold denoising is performed on the first estimation result to obtain the absolute displacement time history of the target cable.

[0029] According to another embodiment of the present invention, a structural vibration monitoring system based on an unmanned aerial vehicle (UAV) is provided, comprising:

[0030] The vibration information acquisition module is used to acquire the vibration information of the target cable, wherein the vibration information of the target cable is obtained after preprocessing the initial vibration information of the target cable;

[0031] The information processing module is used to perform information enhancement and segmentation processing on the target cable vibration information to obtain the processed target cable vibration information;

[0032] The displacement time history processing module is used to extract displacement from the vibration information of the target cable using a preset first algorithm to obtain the displacement time history of the target cable.

[0033] The vibration denoising module is used to perform self-vibration elimination processing on the displacement time history of the target cable using a preset second algorithm to obtain the absolute displacement time history of the target cable.

[0034] The cable force prediction module is used to estimate the cable force of the target cable by using a preset first model to estimate the local displacement time history of the target cable.

[0035] In one exemplary embodiment, the preprocessing of the initial target cable vibration information includes:

[0036] Based on the initial target cable information, an information sequence is determined;

[0037] Noise detection is performed on the information sequence to obtain the position information of the target cable, wherein the position information includes first coordinate information and first coordinate system information;

[0038] Based on the information sequence, the first coordinate information, and the first coordinate system information, a coordinate system transformation process is performed to obtain the position matrix information;

[0039] Based on the position matrix information, the preset target matrix is ​​solved to obtain the target cable vibration information.

[0040] In one exemplary embodiment, the displacement time history processing module includes:

[0041] The displacement coordinate determination unit is used to determine the displacement coordinate information of the target cable based on the vibration information of the target cable, wherein the displacement coordinate information includes second coordinate information and gradient vector information;

[0042] The sub-information determination unit is used to determine the sub-coordinate information and sub-gradient vector information of the target cable based on the second coordinate information;

[0043] The correlation processing unit is used to perform gradient correlation calculation on the gradient vector information and the sub-gradient information, and when the correlation calculation result meets the correlation condition, to perform element point set processing on the second coordinate information and the sub-coordinate information to obtain an initial coordinate point set.

[0044] The coordinate transformation unit is used to perform coordinate transformation calculations based on the initial set of coordinate points to obtain the initial coordinate transformation matrix;

[0045] An update iteration unit is configured to perform update iteration calculations on the initial coordinate transformation matrix based on the initial coordinate point set to obtain a target transformation matrix that satisfies the incremental condition and the corresponding target point set; wherein, the incremental condition includes that the coordinate increment in the sub-coordinate information is less than the target element value; the update iteration calculation includes: minimizing the initial coordinate transformation matrix to obtain a first transformation matrix; performing point set transformation processing on the initial coordinate point set based on the first transformation matrix to obtain a first point set, and updating the target point set with the sub-coordinate information and sub-gradient vector information that are closest to the coordinate points in the initial coordinate point set and the first point set.

[0046] The time history determination unit is used to determine the displacement time history of the target cable based on the target transformation matrix and the target point set.

[0047] In one exemplary embodiment, the vibration denoising module includes:

[0048] The noise estimation unit is used to perform noise estimation processing on the displacement time history of the target cable using the second algorithm to obtain a first estimation result;

[0049] The wavelet denoising unit is used to perform wavelet threshold denoising processing on the first estimation result based on the first coefficient to obtain the absolute displacement time history of the target cable.

[0050] According to yet another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0051] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0052] This invention preprocesses the initial target cable vibration information and uses relevant algorithms for noise reduction, removing coarse UAV displacements and minor vibrations. Information enhancement and segmentation improve the target cable identification accuracy and expand its application range. A preset first model is used to estimate the cable force based on the absolute displacement time history of the target cable, thus obtaining the cable force information. Therefore, this invention solves the problem of self-vibration when using UAVs for structural vibration measurement, thereby achieving accurate identification of structural displacements and precise prediction of cable forces. Attached Figure Description

[0053] Figure 1 This is a hardware structure block diagram of a mobile terminal for a structural vibration monitoring method based on an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0054] Figure 2 This is a flowchart of a structural vibration monitoring method based on an unmanned aerial vehicle (UAV) according to an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of vibration information acquisition according to an embodiment of the present invention;

[0056] Figure 4 This is a schematic diagram of PINN model training according to an embodiment of the present invention;

[0057] Figure 5 This is a schematic diagram of an infrared marker according to an embodiment of the present invention;

[0058] Figure 6 Based on the embodiments of the present invention Figure 5 A diagram illustrating tag matching;

[0059] Figure 7 This is a schematic diagram of the correlation calculation process according to an embodiment of the present invention;

[0060] Figure 8This is a structural block diagram of a structure vibration monitoring system based on an unmanned aerial vehicle (UAV) according to an embodiment of the present invention. Detailed Implementation

[0061] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples.

[0062] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0063] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a cable force detection method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0064] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to a UAV-based structural vibration monitoring method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the aforementioned method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0065] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0066] This embodiment provides a cable tension detection method. Figure 2 This is a flowchart of a cable tension detection method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0067] Step S201: Obtain target cable vibration information, wherein the target cable vibration information is obtained after preprocessing the initial target cable vibration information;

[0068] In this embodiment, when the cable is taut under stress, it will produce small vibrations in some areas. By detecting these vibrations, the stress condition of the cable can be determined. Preprocessing is performed to remove coarse displacement / coarse vibrations from the UAV (i.e., the device carrying the detection equipment), thereby reducing interference in subsequent processing and improving the accuracy of cable stress detection.

[0069] The target cable can be a bridge cable located in a dim environment (such as a cable in a bridge engineering building), or a bridge or engineering cable in a relatively bright environment (such as a cable exposed to the outdoors), or other types of cables; the initial target cable vibration information can be (but is not limited to) the sound wave feedback information obtained by ultrasonic signal detection of the cable, or the vibration signal obtained by other means, or the image information of cable vibration marked by infrared laser.

[0070] Taking image information acquisition as an example, the specific data acquisition method is as follows:

[0071] An infrared laser beam fixed beside the cable-stayed bridge is used to illuminate and mark the cables. Since the laser emission point is located on the ground below the bridge, the marked points on the cables are considered fixed points. A drone is used to capture video of the cable vibration at the infrared marked points against a low-light background, thus obtaining... Figure 3 The video information shown.

[0072] Step S202: Perform information enhancement and segmentation processing on the target cable vibration information to obtain the processed target cable vibration information;

[0073] In this embodiment, in some special environments, the vibration of the cable is not easily observed due to environmental interference (for example, in a dim environment, image data is not easy to identify; or in a noisy environment, ultrasonic detection is prone to being mixed with a lot of noise). Therefore, it is necessary to perform information enhancement and information segmentation for the relevant environment to facilitate the accurate identification of the content to be detected in the future.

[0074] Among them, information enhancement and segmentation processing can (but is not limited to) using neural networks to enhance ultrasonic signals in noisy environments and perform region clustering and segmentation on the enhanced feedback signals to obtain signal sequences with relatively prominent signal waveforms; it can also use neural network algorithms to perform low-light enhancement on cables in dark backgrounds to obtain cable images with ideal backgrounds and segment the cable edges to achieve images where the cable edges are clearly distinguishable from the background images; or it can be other enhancement methods.

[0075] Specifically, taking image information processing as an example, the processes of image information enhancement and segmentation can be achieved in the following ways:

[0076] The transformed Lasso images were divided into two categories: bright backgrounds and dark backgrounds. They were then labeled to form a dataset of 6000 pairs, with 60% used as the training set, 10% as the test set, and 30% as the validation set.

[0077] Subsequently, two generators, G1 and G2, are built based on the U-Net network, and two discriminators, D1 and D2, are built based on Enlighten-GAN. The image with a dark background is segmented into a 256×256 resolution. G1 performs four downsampling operations on the input image to obtain a 16×16×1024 feature map. Then, the feature map is upsampled four times to obtain the transformed 256×256 resolution image. The dark background image and the bright background image generated by the generator are convolved five times to obtain a 16×16×1 matrix. The average of the matrix is ​​used to obtain the discrimination result of D1. G1 is used to generate the bright background image, and D1 is used to distinguish between the bright background image generated by G1 and the real bright background image.

[0078] The image generated by G1 is then input into G2, and similarly, four downsampling and upsampling operations are performed. The cable edge contour image generated by G2 is then convolved five times by D2 to obtain a 16×16×1 matrix. The average of this matrix yields the D2 discrimination result. G2 is used to automatically generate the cable edge contour to clearly distinguish it from the background image, facilitating the next step of displacement extraction. D2 is used to differentiate the cable edge contour image generated by G2 from the bright background image.

[0079] Finally, the loss function of the discriminator is established, as shown in equation (1):

[0080]

[0081] Step S203: Extract displacement from the vibration information of the target cable using a preset first algorithm to obtain the displacement time history of the target cable;

[0082] In this embodiment, during the vibration of the cable under stress, a small amount of displacement will occur locally. At this time, the cable force can be determined by the change in displacement. The first algorithm is used to determine the displacement change of the cable without manual processing, and the displacement data can be determined more accurately.

[0083] The first algorithm can be (but is not limited to) an adaptive beamforming algorithm for ultrasonic detection, or a sub-pixel adaptive edge point matching (SEPM) algorithm for image information; the displacement time history includes pixel displacement or acoustic displacement data of the cable in the image or in the acoustic feedback under the action of cable force.

[0084] Step S204: The displacement time history of the target cable is processed by the self-vibration elimination of the UAV through a preset second algorithm to obtain the absolute displacement time history of the target cable;

[0085] In this embodiment, noise reduction processing is performed on the vibration noise of the detection device itself contained in the displacement time history response information to avoid the influence of the vibration of the detection device itself on the detection results.

[0086] The second algorithm can be (but is not limited to) a mode decomposition algorithm such as Empirical Mode Decomposition (EMD) and Singular Value Decomposition (SVD), a wavelet transform denoising algorithm, or other denoising algorithms.

[0087] Step S205: Estimate the cable force of the target cable by using a preset first model to estimate the absolute displacement time history of the target cable.

[0088] In this embodiment, the accurate cable force detection results can be obtained quickly by predicting and calculating the denoised absolute displacement time history using a neural network model, thereby improving detection efficiency.

[0089] The first model can be a pre-trained PINN model of cable tension, and the specific training process can be as follows: Figure 4 As shown, the stiffness of the cable is corrected by the following equation (3). The physical equations of the cable stiffness, inclination angle and other parameters are added to the PINN network to fit the redundant constraints. The various parameters of the cable are input to train the PINN network until the loss function converges.

[0090]

[0091]

[0092]

[0093] The parameters in equations (2) to (4) are explained as follows: w is the weight per unit cable length, g is the gravitational acceleration, y is the transverse coordinate perpendicular to the cable length direction, t is time, x is the longitudinal coordinate perpendicular to the cable length direction, T is the cable tension, EI is the cable bending stiffness, E is the elastic modulus perpendicular to the cable, and E i L is the modified elastic modulus of the cable, and L is the cable length. x Let be the horizontal projected length of the cable, A be the cross-sectional area of ​​the cable, α be the inclination angle of the cable, G be the weight of the cable, and H be the horizontal component of the force on the cable. Figure 4 middle Let N be the loss function of equation (4), and N be the nonlinear differential operator. Let W be the loss function of equation (2), W be the weight of the neural network, and b be the bias of the neural network.

[0094] Through the above steps, by preprocessing the initial target cable vibration information and denoising it using relevant algorithms, coarse displacements and minor vibrations of the detection equipment itself can be removed. Information enhancement and segmentation processing can improve the identification accuracy of the target cable and expand its application range. Cable force prediction using the first model can effectively avoid the problem that traditional frequency-based cable force prediction is easily affected by cable boundary conditions, resulting in higher accuracy of the predicted cable force. Therefore, it can solve the problem of low cable force detection accuracy and improve the accuracy and efficiency of cable force detection.

[0095] The entities that perform the above steps can be base stations, terminals, etc., but are not limited to these.

[0096] In an optional embodiment, the preprocessing of the initial target cable vibration information includes:

[0097] Step S2011: Determine the information sequence based on the initial target cable information;

[0098] Step S2012: Noise detection is performed on the information sequence to obtain the position information of the target cable, wherein the position information includes first coordinate information and first coordinate system information;

[0099] Step S2013: Based on the information sequence, the first coordinate information, and the first coordinate system information, perform coordinate system transformation processing to obtain position matrix information;

[0100] Step S2014: Based on the position matrix information, solve the preset target matrix to obtain the target cable vibration information.

[0101] In this embodiment, the information sequence can be (but is not limited to) a sound wave sequence of sound wave signals, or an image frame sequence of image information; correspondingly, noise detection can be image spot detection of image frames, filtering detection of ultrasonic signals, or corresponding noise detection for other types of data; correspondingly, the coordinate information includes the sequence position information of a certain data element in the sequence information, such as the coordinates of an element point in an image frame, or the position coordinates of a certain sound wave signal point in the sound wave array in the sound wave signal sequence, and so on; the position matrix information includes a position matrix composed of the spatial coordinates and orientation angles of related elements (as shown in formulas (9)-(10)); the coordinate relationship transformation processing includes establishing a relationship between the coordinates in the sequence and the spatial coordinates in real time (as shown in formulas (14)-(17)) to achieve pose unification of different frame image sequences or sound wave sequences and eliminate the large range of self-vibration (i.e. coarse vibration) of the shooting device itself; the first coordinate system information includes the coordinate system information of the elements in the sequence and the spatial coordinate system information in reality.

[0102] Taking blob detection for image information as an example, the processing steps include:

[0103] Spot detection is performed using the scale-normalized Laplace of Gaussian (LOG) operator. The normalized LOG operator is shown in Equation (5). Given the radius of the marker point, its characteristic scale σ can be obtained, thus realizing the localization and tracking of the infrared marker point. Figures 5-6 As shown.

[0104] The feature information of the marker point is obtained by calculating the horizontal gradient, vertical gradient and gradient direction angle, and its expression is shown in equations (6)(7)(8).

[0105]

[0106]

[0107]

[0108]

[0109]

[0110]

[0111] Where: x and y are the pixel coordinates of the image, σ is the feature scale of the image, f(x,y) is the two-dimensional grayscale function of the image frame, and I x ,I y These are the Sobel operators in the x and y directions (corresponding to the aforementioned position matrix information), and θ is the direction angle of the gradient.

[0112] Subsequently, key point descriptor sets are established for each adjacent frame. The similarity of the descriptors is measured by equation (13), and abnormal matching point pairs are removed. The feature point matching effect is as follows: Figure 5 As shown.

[0113] The pixel coordinate system (corresponding to the aforementioned coordinate system information) and the world coordinate system are transformed using a linear transformation method (corresponding to the aforementioned coordinate system transformation process), as shown in equations (14) and (15). By substituting the four coordinate points of the infrared marker into equation (18) and solving the equations simultaneously, the homography matrix (corresponding to the aforementioned target matrix) can be solved, thereby achieving pose unification for different frame image sequences and eliminating the self-vibration of the UAV over a large range.

[0114] R i =(r i1 ,r i2 ,···,r i8 (11)

[0115] S i =(s i1 ,s i2 ,···,s i8 (12)

[0116]

[0117]

[0118]

[0119] Where: R i S i This is a subset of keypoint descriptions for adjacent frames, where (u,v) are the planar pixel coordinates of the corresponding 3D point on the image (corresponding to the aforementioned first coordinate information), and d x The width d in the x-direction for each pixel y The width of each pixel in the y-direction, (x w ,y w ,z w (x) represents the three-dimensional world coordinates. c ,y c ,z c Let R be the camera coordinate system, R be the rotation matrix, t be the translation vector, K be the camera intrinsic parameters, and f be the translation vector. x ,f yThe focal lengths in the x and y directions are respectively, c x ,c y This is the offset of the optical axis relative to the coordinate center of the projection plane.

[0120]

[0121] Let a 33 =1, then we have

[0122] Where (x1,y1) and (x2,y2) are the image coordinates before and after the coordinate transformation, respectively, and H is the homography matrix.

[0123] In an optional embodiment, the step of extracting displacement from the vibration information of the target cable using a preset first algorithm to obtain the displacement time history of the target cable includes:

[0124] Step S2031: Based on the vibration information of the target cable, determine the displacement coordinate information of the target cable, wherein the displacement coordinate information includes second coordinate information and gradient vector information;

[0125] In this embodiment, vibration information from a certain sequence is selected, and then the corresponding element coordinate information and gradient vector information are determined by a specific algorithm, thereby facilitating subsequent accurate calculations.

[0126] For acoustic wave detection, the gradient vector information includes the acoustic wave attenuation gradient in a certain direction. For image frame sequences, the gradient vector information includes the slope of pixel changes in the image. For example, consider an image sequence:

[0127] Using the cable in the first frame as the initial target, and applying non-maximum suppression (NMS) to locate the moving target in subsequent frames at integer pixel resolution, the pixel coordinates p of the edge points within the Region of Interest (ROI) can be obtained. i =(x i ,y i (corresponding to the aforementioned second coordinate information), gradient vector d i =(d xi ,d yi (corresponding to the aforementioned gradient vector information).

[0128] Step S2032: Based on the second coordinate information, determine the sub-coordinate information and sub-gradient vector information of the target cable;

[0129] In this embodiment, after obtaining the coordinate information and gradient vector corresponding to the relevant elements, the changed coordinate information and gradient vector are then determined to facilitate the determination of the changes.

[0130] The relevant elements can be pixels, sound wave points, or elements from other detection methods. Taking image detection as an example, the process of determining sub-coordinate information and sub-gradient vector information is as follows:

[0131] Fit p using a second-order polynomial. i The gray intensity within a 3×3 neighborhood around the point will be p i The extreme point coordinates of a point polynomial function are considered as its sub-pixel coordinates, denoted by p. subi =(x subi ,y subi The coordinates q of potential edge points of the moving target within the ROI region are calculated by applying the nonmaximum suppression (NMS) technique in the Canny operator. j =(u j ,v j The sub-pixel coordinates q are calculated using a second-order polynomial fitting technique. subj =(u subj ,v subj (corresponding to the aforementioned sub-coordinate information).

[0132] Assume the horizontal and vertical displacements of the cable are D respectively. x D y , then u j =x i +D x v j =y i +D y Initial point integer pixel coordinates p i =(x i ,y i The corresponding point in subsequent frames can be represented by q. i =(u i ,v i The subgradient vector is represented as e = (e ui ,e vi (corresponding to the aforementioned sub-gradient vector information), initial frame sub-pixel coordinates p subi =(x subi ,y subi The corresponding point in subsequent frames can be represented by q. subi =(u subi ,v subi )express.

[0133] Step S2033: Perform gradient correlation calculation on the gradient vector information and the sub-gradient information, and if the correlation calculation result satisfies the correlation condition, perform element point set processing on the second coordinate information and the sub-coordinate information to obtain an initial coordinate point set.

[0134] In this embodiment, to ensure the accuracy of the detection, it is necessary to determine the correlation between the relevant data before and after the change.

[0135] The correlation can be calculated using the corresponding correlation calculation formula. Taking image detection as an example, it can be calculated using... and Quantification is performed, if and Then p can be subi and q subi The extracted points are placed into the initial target and moving target point sets (corresponding to the aforementioned initial coordinate point set); otherwise, sub-pixel coordinates will not be extracted. and The threshold for these two correlation coefficients, and The calculation methods are as follows: equations (10) and (11), e avg and d avg The average value of the gradient vector is used to account for variations in ambient lighting.

[0136]

[0137]

[0138] Step S2034: Based on the initial set of coordinate points, perform coordinate transformation calculations to obtain the initial coordinate transformation matrix;

[0139] In this embodiment, the set of coordinate points is transformed into a corresponding matrix, which facilitates the calculation of complex data and makes it easier to observe its correlation.

[0140] For example, taking image detection as an example, coordinate transformation calculation includes:

[0141] The sub-pixel coordinate sets of the extracted initial target and moving target can be represented as follows: and The coordinate transformation between the initial target and the moving target can be expressed as: Here λ x and λ y For the horizontal and vertical displacements, their initial values ​​correspond to D = (D x D y ), θ is the rotation angle, its initial value is 0, and is the sub-pixel coordinate of the moving target. Therefore, the objective function equation φ(ω) can be established, as shown in equation (21):

[0142]

[0143] Step S2035: Based on the initial coordinate point set, perform update iteration calculation on the initial coordinate transformation matrix to obtain a target transformation matrix that satisfies the incremental condition and the corresponding target point set; wherein, the incremental condition includes that the coordinate increment in the sub-coordinate information is less than the target element value; the update iteration calculation includes: minimizing the initial coordinate transformation matrix to obtain a first transformation matrix; performing point set transformation processing on the initial coordinate point set based on the first transformation matrix to obtain a first point set, and updating the target point set with the sub-coordinate information and sub-gradient vector information that are closest to the coordinate points in the initial coordinate point set and the first point set.

[0144] In this embodiment, the coordinate matrix of the initial coordinate point set is continuously updated and iterated to determine the displacement transformation corresponding to the smallest element, so as to improve the detection accuracy.

[0145] Taking image detection as an example, the corresponding update iteration process includes: minimizing φ(ω) to obtain the new coordinate transformation matrix. Transform the initial target point set into P * =Pω new closest potential edge point coordinates q subj To be extracted Update within the point set. ω new and Q new Iterative updates are performed using the above method, when two increments λ... xnew and λ ynew When the difference is less than 0.01 pixels (corresponding to the aforementioned increment condition), the iteration stops, and the sub-pixel displacement of the target is (λ). xnew -D x ,λ ynew -D y The subpixel displacement extraction flowchart is as follows: Figure 7 As shown.

[0146] Step S2036: Determine the displacement time history of the target cable based on the target transformation matrix and the target point set.

[0147] In an optional embodiment, the step of performing UAV self-vibration elimination processing on the displacement time history of the target cable using a preset second algorithm to obtain the absolute displacement time history of the target cable includes:

[0148] Step S2041: The displacement time history of the target cable is subjected to noise estimation processing using the second algorithm to obtain a first estimation result;

[0149] Step S2042: Perform wavelet threshold denoising processing on the first estimation result based on the first coefficient to obtain the absolute displacement time history of the target cable.

[0150] In this embodiment, the second algorithm may be (but is not limited to) the Complete EEMD with Adaptive Noise (CEEMDAN) algorithm, and the wavelet threshold denoising algorithm may be (but is not limited to) using the introduction of Pearson correlation coefficient (as shown in Equation (22)) and wavelet soft threshold denoising method to determine the effective information contained in each order of Intrinsic Mode Function (IMF). By comparing the changes in Pearson correlation coefficient before and after denoising, effective IMFs are distinguished and irrelevant intrinsic mode components are eliminated, thereby eliminating the vibration of the device itself mounted on the detection device.

[0151] Specifically, taking image detection as an example, the wavelet basis type used for wavelet threshold denoising is sym8, and the decomposition layer is 4. Since the frequency of the vibration of the UAV equipped with the infrared detection device is low, it is more appropriate to use a fixed threshold estimation method for denoising. At the same time, an improved wavelet soft threshold function is proposed (as shown in Equation (23)). Compared with the traditional soft threshold and hard threshold functions, the denoised signal has better smoothness and continuity, and can avoid the generation of fixed bias.

[0152] After wavelet decomposition and soft thresholding, the retained IMFs are reconstructed to obtain a smooth reconstructed signal, eliminating low-order UAV vibrations in the cable displacement signal.

[0153]

[0154]

[0155] Where: x represents the original signal, c represents the intrinsic mode components after CEEMDAN decomposition, conv(c,x) is the covariance, and σ x Let be the standard deviation, and sgn(x) be the sign function, whose expression is: α and β are adjustment factors, and t is the threshold.

[0156] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0157] This embodiment also provides a UAV-based structural vibration monitoring system, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0158] Figure 8 This is a structural block diagram of a UAV-based structural vibration monitoring system according to an embodiment of the present invention, such as... Figure 8 As shown, the device includes:

[0159] The vibration information acquisition module 81 is used to acquire the vibration information of the target cable, wherein the vibration information of the target cable is obtained after preprocessing the initial vibration information of the target cable;

[0160] The information processing module 82 is used to perform information enhancement and segmentation processing on the target cable vibration information to obtain the processed target cable vibration information;

[0161] The displacement time history processing module 83 is used to extract displacement from the vibration information of the target cable through a preset first algorithm to obtain the displacement time history of the target cable;

[0162] The vibration denoising module 84 is used to perform self-vibration elimination processing on the displacement time history of the target cable using a preset second algorithm to obtain the absolute displacement time history of the target cable.

[0163] The cable force prediction module 85 is used to estimate the cable force of the target cable by using a preset first model to estimate the local displacement time history of the target cable.

[0164] In one exemplary embodiment, the preprocessing of the initial target cable vibration information includes:

[0165] Based on the initial target cable information, an information sequence is determined;

[0166] Noise detection is performed on the information sequence to obtain the position information of the target cable, wherein the position information includes first coordinate information and first coordinate system information;

[0167] Based on the information sequence, the first coordinate information, and the first coordinate system information, a coordinate system transformation process is performed to obtain the position matrix information;

[0168] Based on the position matrix information, the preset target matrix is ​​solved to obtain the target cable vibration information.

[0169] In one exemplary embodiment, the displacement time history processing module 83 includes:

[0170] The displacement coordinate determination unit is used to determine the displacement coordinate information of the target cable based on the vibration information of the target cable, wherein the displacement coordinate information includes second coordinate information and gradient vector information;

[0171] The sub-information determination unit is used to determine the sub-coordinate information and sub-gradient vector information of the target cable based on the second coordinate information;

[0172] The correlation processing unit is used to perform gradient correlation calculation on the gradient vector information and the sub-gradient information, and when the correlation calculation result meets the correlation condition, to perform element point set processing on the second coordinate information and the sub-coordinate information to obtain an initial coordinate point set.

[0173] The coordinate transformation unit is used to perform coordinate transformation calculations based on the initial set of coordinate points to obtain the initial coordinate transformation matrix;

[0174] An update iteration unit is configured to perform update iteration calculations on the initial coordinate transformation matrix based on the initial coordinate point set to obtain a target transformation matrix that satisfies the incremental condition and the corresponding target point set; wherein, the incremental condition includes that the coordinate increment in the sub-coordinate information is less than the target element value; the update iteration calculation includes: minimizing the initial coordinate transformation matrix to obtain a first transformation matrix; performing point set transformation processing on the initial coordinate point set based on the first transformation matrix to obtain a first point set, and updating the target point set with the sub-coordinate information and sub-gradient vector information that are closest to the coordinate points in the initial coordinate point set and the first point set.

[0175] The time history determination unit is used to determine the displacement time history of the target cable based on the target transformation matrix and the target point set.

[0176] In one exemplary embodiment, the vibration denoising module 84 includes:

[0177] The noise estimation unit is used to perform noise estimation processing on the displacement time history of the target cable using the second algorithm to obtain a first estimation result;

[0178] The wavelet denoising unit is used to perform wavelet threshold denoising processing on the first estimation result based on the first coefficient to obtain the absolute displacement time history of the target cable.

[0179] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0180] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0181] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0182] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0183] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0184] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0185] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0186] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A structural vibration monitoring method based on unmanned aerial vehicles (UAVs), characterized in that, include: The target cable vibration information is obtained by preprocessing the initial target cable vibration information. The target cable vibration information is subjected to information enhancement and segmentation processing to obtain the processed target cable vibration information; Displacement is extracted from the vibration information of the target cable using a preset first algorithm to obtain the displacement time history of the target cable; The displacement time history of the target cable is processed by the UAV's self-vibration elimination through a preset second algorithm to obtain the absolute displacement time history of the target cable. The cable force of the target cable is estimated by performing the absolute displacement time history of the target cable using a preset first model. The step of extracting displacement from the vibration information of the target cable using a preset first algorithm to obtain the displacement time history of the target cable includes: Based on the vibration information of the target cable, the displacement coordinate information of the target cable is determined, wherein the displacement coordinate information includes second coordinate information and gradient vector information; Based on the second coordinate information, the sub-coordinate information and sub-gradient vector information of the target cable are determined; Gradient correlation is calculated on the gradient vector information and the sub-gradient vector information. If the correlation calculation result satisfies the correlation condition, the second coordinate information and the sub-coordinate information are processed into element point set to obtain an initial coordinate point set. Based on the initial set of coordinate points, coordinate transformation calculations are performed to obtain the initial coordinate transformation matrix; Based on the initial coordinate point set, an update iteration calculation is performed on the initial coordinate transformation matrix to obtain a target transformation matrix that satisfies the incremental condition and the corresponding target point set; wherein, the incremental condition includes that the coordinate increment in the sub-coordinate information is less than the target element value; the update iteration calculation includes: minimizing the initial coordinate transformation matrix to obtain a first transformation matrix; performing point set transformation processing on the initial coordinate point set based on the first transformation matrix to obtain a first point set, and updating the target point set with the sub-coordinate information and sub-gradient vector information that are closest to the coordinate points in the initial coordinate point set and the first point set; The displacement time history of the target cable is determined based on the target transformation matrix and the target point set. The step of performing self-vibration elimination processing on the displacement time history of the target cable using a preset second algorithm to obtain the absolute displacement time history of the target cable includes: The displacement time history of the target cable is subjected to noise estimation processing using the second algorithm to obtain a first estimation result; Based on the first coefficient, wavelet threshold denoising is performed on the first estimation result to obtain the absolute displacement time history of the target cable.

2. The method according to claim 1, characterized in that, The preprocessing of the initial target cable vibration information includes: Based on the initial target cable vibration information, an information sequence is determined; Noise detection is performed on the information sequence to obtain the position information of the target cable, wherein the position information includes first coordinate information and first coordinate system information; Based on the information sequence, the first coordinate information, and the first coordinate system information, a coordinate system transformation process is performed to obtain the position matrix information; Based on the position matrix information, the preset target matrix is ​​solved to obtain the target cable vibration information.

3. A structural vibration monitoring system based on unmanned aerial vehicles (UAVs), characterized in that, include: The vibration information acquisition module is used to acquire the vibration information of the target cable, wherein the vibration information of the target cable is obtained after preprocessing the initial vibration information of the target cable; The information processing module is used to perform information enhancement and segmentation processing on the target cable vibration information to obtain the processed target cable vibration information; The displacement time history processing module is used to extract displacement from the vibration information of the target cable using a preset first algorithm to obtain the displacement time history of the target cable. The vibration denoising module is used to perform self-vibration elimination processing on the displacement time history of the target cable using a preset second algorithm to obtain the absolute displacement time history of the target cable. The cable force prediction module is used to estimate the cable force of the target cable by using a preset first model to estimate the local displacement time history of the target cable. The displacement time history processing module includes: The displacement coordinate determination unit is used to determine the displacement coordinate information of the target cable based on the vibration information of the target cable, wherein the displacement coordinate information includes second coordinate information and gradient vector information; The sub-information determination unit is used to determine the sub-coordinate information and sub-gradient vector information of the target cable based on the second coordinate information; The correlation processing unit is used to perform gradient correlation calculation on the gradient vector information and the sub-gradient vector information, and when the correlation calculation result meets the correlation condition, to perform element point set processing on the second coordinate information and the sub-coordinate information to obtain an initial coordinate point set. The coordinate transformation unit is used to perform coordinate transformation calculations based on the initial set of coordinate points to obtain the initial coordinate transformation matrix; An update iteration unit is configured to perform update iteration calculations on the initial coordinate transformation matrix based on the initial coordinate point set to obtain a target transformation matrix that satisfies the incremental condition and the corresponding target point set; wherein, the incremental condition includes that the coordinate increment in the sub-coordinate information is less than the target element value; the update iteration calculation includes: minimizing the initial coordinate transformation matrix to obtain a first transformation matrix; performing point set transformation processing on the initial coordinate point set based on the first transformation matrix to obtain a first point set, and updating the target point set with the sub-coordinate information and sub-gradient vector information that are closest to the coordinate points in the initial coordinate point set and the first point set. The time history determination unit is used to determine the displacement time history of the target cable based on the target transformation matrix and the target point set; The vibration noise reduction module includes: The noise estimation unit is used to perform noise estimation processing on the displacement time history of the target cable using the second algorithm to obtain a first estimation result; The wavelet denoising unit is used to perform wavelet threshold denoising processing on the first estimation result based on the first coefficient to obtain the absolute displacement time history of the target cable.

4. The system according to claim 3, characterized in that, The preprocessing of the initial target cable vibration information includes: Based on the initial target cable vibration information, an information sequence is determined; Noise detection is performed on the information sequence to obtain the position information of the target cable, wherein the position information includes first coordinate information and first coordinate system information; Based on the information sequence, the first coordinate information, and the first coordinate system information, a coordinate system transformation process is performed to obtain the position matrix information; Based on the position matrix information, the preset target matrix is ​​solved to obtain the target cable vibration information.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is configured to perform the method described in any one of claims 1 to 2 when executed.

6. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method as described in any one of claims 1 to 2.

Images