A bridge sling complex background elimination, micro-vibration displacement robust extraction and super-resolution frequency identification method based on unmanned aerial vehicle vision

Abstract

The application relates to a bridge sling complex background elimination, micro-vibration displacement robust extraction and super-resolution frequency identification method based on unmanned aerial vehicle vision, which comprises the following steps: establishing an outer rectangle ROI and an inner parallelogram ROI in a video image; performing non-interpolation integral pixel sampling slicing on the inner parallelogram ROI; extracting gradient features and performing nonlinear logarithmic gradient compression; performing template matching on multiple one-dimensional sections in parallel, and performing median consensus processing to obtain an initial displacement time history of the sling; performing LOWESS adaptive detrending processing; performing filter bank random decrement processing; determining a real structure modal band based on an oscillation cycle number physical criterion and an energy-purity score; and performing MUSIC super-resolution spectrum estimation to obtain a sling vibration frequency. Compared with the prior art, the application can realize stable extraction of sling displacement and high-precision identification of the frequency under the conditions of a complex urban background, a low-texture rope body, small-amplitude vibration, a low-frame-rate short video and unmanned aerial vehicle hovering disturbance.

Description

Technical Field

[0001] This invention relates to the field of bridge structural health monitoring technology, and in particular to a method for eliminating complex backgrounds of bridge suspension cables, robustly extracting micro-vibration displacements, and super-resolution frequency recognition based on UAV vision. Background Technology

[0002] The cables in long-span arch bridges and suspension bridges are critical load-bearing components, and their stress state directly affects the overall stress distribution and operational safety of the bridge. Traditional cable stress detection typically uses accelerometers to obtain the cable vibration frequency and then calculates the cable force using string vibration theory or a modified model. This method requires manual installation of sensors near the cables, which presents problems such as high operational risks, low detection efficiency, and difficulty in routine deployment.

[0003] UAV-based airborne vision measurement offers advantages such as non-contact operation, long-range capability, high maneuverability, and integration with automated hangars, making it suitable for bridge cable inspection. However, in urban bridge scenarios, cables are often interwoven with strong vertical structural backgrounds such as building window frames, glass curtain walls, and railings, resulting in background gradient amplitudes that are often much larger than the surface texture of the cables. Furthermore, the cable surface is typically smooth rubber or polyethylene sheathing, lacking corner points and stable local textures, causing conventional optical flow, edge detection, and template matching methods to easily drift into the background area.

[0004] Furthermore, when drones hover at close range, they are affected by rotor airflow, gusts, and aerodynamic disturbances, resulting in high-energy, low-frequency rigid body drift. The amplitude of the sling's micro-vibration is very small, and its true fundamental frequency may be masked by the drone's drift spectrum. In addition, most airborne videos have low frame rates, limiting the highest measurable frequency, and short time windows are often used to ensure signal stability. This severely limits traditional spectrum analysis to Rayleigh frequency resolution, making it difficult to meet the accuracy requirements of engineering frequency pickup and cable force identification.

[0005] Several vision-based frequency and recognition technologies exist, such as patent CN115144102B—an automatic cruise monitoring system and method for bridge cable force based on a PTZ camera—which fails to address the issues of extracting micro-vibrations of suspension cables in complex backgrounds and achieving super-resolution frequency recognition in low frame rate videos; patent CN115876371B—an optical measurement method and system for cable force based on multi-mode shape fitting, which relies on fixed-point camera shooting and cannot cover scenarios where interference caused by UAV displacement needs to be eliminated; and patent application CN118067294A—a computer vision-based method for detecting cable force in cable-stayed bridges, which uses edge detection for cable-stayed cables but cannot cover cable edges and complex background edges. The following patents apply: CN116363121B – A method, system, and device for detecting cable force based on computer vision; CN117456422A – A method for testing cable force based on infrared light and optimal adaptive template matching; CN119942322A – A method and device for real-time cable force identification based on deep learning. These patents use single template matching or optical flow to identify the cable force of inclined cables, but fail to cover scenarios involving complex background cable frequency extraction and super-resolution frequency identification; CN121163728A – An efficient method for identifying bridge cable force using a drone target, which requires setting a target and does not cover scenarios without a target.

[0006] Therefore, there is a need for a UAV visual measurement technology for bridge cable displacement and frequency identification that can simultaneously address the challenges of complex backgrounds, small amplitudes, and low frame rates with short data. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a robust extraction and super-resolution frequency identification method for micro-vibration displacement of bridge slings under complex backgrounds that can effectively suppress low-frequency high-energy interference from UAVs and achieve high accuracy in sling frequency identification.

[0008] The objective of this invention can be achieved through the following technical solutions: A method for complex background removal, robust extraction of micro-vibration displacement, and super-resolution frequency recognition of bridge suspension cables based on UAV vision includes the following steps: Acquire video images containing the suspension cables of the bridge under test; An outer rectangular region of interest and an inner parallelogram region of interest are established in the video image. The inner parallelogram region of interest is generated based on the line connecting the four corner points of a local segment of the sling, after parallelizing the opposite sides, and is located within the outer rectangular region of interest. The region of interest of the inner parallelogram is sampled and sliced ​​without interpolation according to a preset channel step size to obtain multiple one-dimensional cross sections distributed along the length of the suspension cable. Gradient features are extracted from each one-dimensional cross section and nonlinear logarithmic gradient compression is performed. Based on the compressed gradient features, template matching is performed in parallel on multiple one-dimensional cross sections to obtain displacement estimates for each channel; The displacement estimates of each channel are processed by median consensus, and the initial displacement time history of the sling is obtained by sub-pixel peak interpolation. The initial displacement time history is subjected to LOWESS adaptive detrending processing to obtain the true micro-vibration displacement time history of the sling; The actual micro-vibration displacement time history of the sling is subjected to random reduction processing of the filter bank to obtain the equivalent free attenuation response corresponding to multiple candidate frequency bands; Based on the equivalent free decay response corresponding to each candidate frequency band, the true structural mode frequency band is determined according to the oscillation cycle number physics criterion and energy-purity score; The equivalent free attenuation response corresponding to the actual structural modal frequency band is subjected to MUSIC super-resolution spectral estimation to obtain the cable vibration frequency.

[0009] Furthermore, the outer rectangular region of interest is used to define the maximum possible vibration envelope region of the sling.

[0010] Furthermore, the interpolation-free integer pixel sampling slice includes: Along the left and right boundaries of the region of interest of the inner parallelogram, the horizontal coordinates of the left and right boundaries are calculated based on the current cross-sectional ordinate. The calculated sampling coordinates are rounded and pixels are read from the original video image to obtain the one-dimensional cross-section.

[0011] Furthermore, the horizontal edge gradient of the one-dimensional cross-section is calculated as the gradient feature.

[0012] Furthermore, the template matching includes: Using the compressed gradient features of the first frame of video image as a compressed gradient template, normalized cross-correlation template matching is performed on the compressed gradient features corresponding to the local search region of each channel in subsequent frames to obtain the discrete displacement estimate of each channel.

[0013] Furthermore, the nonlinear logarithmic gradient compression takes the following form: the original gradient function of the one-dimensional cross-section g ( x ), x Calculate the compression gradient using the x-coordinate of the index. ,in, For symbolic functions, It is the natural logarithm function.

[0014] Furthermore, the median consensus processing includes: Calculate the median of the displacement estimates for all channels in the current frame; When the deviation between the displacement estimate of a certain channel and the median exceeds the preset consensus tolerance, the channel is identified as an abnormal channel and its current measurement value is discarded. Force the search center of the next frame of the abnormal channel to be corrected to the position corresponding to the median; The measured values ​​of the normal channels are retained and the search center is updated in the next frame. Finally, the median of the displacement estimates of all channels in the current frame is used as the global high-robust displacement output of the system for the current frame.

[0015] Furthermore, the subpixel peak interpolation is obtained by performing logarithmic parabolic interpolation on the cross-correlation peak points and their left and right adjacent points. The logarithmic parabolic interpolation is based on the assumption that the neighborhood of the cross-correlation peaks follows a Gaussian distribution and is used to convert the discrete pixel-level displacement estimation into a subpixel-level displacement time history, which serves as the initial displacement time history of the sling.

[0016] Furthermore, the smoothing parameters of the LOWESS adaptive detrending process are adaptively determined based on the signal characteristics of the initial displacement time history. The determination process includes: The amplitude spectrum is obtained by performing a Fourier transform on the initial displacement time history, and the highest peak value of the non-zero frequency is extracted as the candidate value of the predicted fundamental frequency. ; Preset suspension cable theoretical fundamental frequency range ,like If the frequency falls within the range, it will be used as the estimated fundamental frequency. ;like If the value exceeds the upper limit of the interval, divide it successively by positive integers greater than 1 until the calculation result falls within the theoretical fundamental frequency interval, thus obtaining the estimated fundamental frequency. ; Set equivalent cutoff frequency for 1 / 3 to 1 / 2, according to The smoothing parameters are solved in reverse so that the trend term only includes the low-frequency drift of the UAV and the residual term retains the sling vibration component.

[0017] Furthermore, the filter bank random decrementing process includes: The target frequency band is divided into multiple overlapping narrow bands. After filtering the signal of each narrow band, a preset multiple of the standard deviation of the narrow band signal is used as the random decrement trigger threshold. The triggering start point is taken only when the narrow band signal crosses the trigger threshold from bottom to top. Response segments of fixed length are continuously intercepted and averaged to obtain the equivalent free attenuation response of the corresponding narrow band.

[0018] Furthermore, determining the true structural modal frequency band includes: The number of positive local maxima in the equivalent free decay response of the candidate frequency band within a preset time window is counted. If the number of positive local maxima is less than a preset threshold, the candidate frequency band is determined to be low-frequency drift or unstructured interference of UAV or other airborne equipment and is removed. Otherwise, the corresponding candidate frequency band is determined to pass the physical criterion of oscillation cycle number. The power spectral density of candidate frequency bands that pass the physical criterion of oscillation cycle number is calculated. The spectral purity is determined based on the ratio of the maximum power spectral density in the frequency band to the total power spectral density of the frequency band. The product of the spectral purity and the absolute energy of the frequency band is used as a comprehensive score. The candidate frequency band with the highest comprehensive score is selected as the true structural mode frequency band.

[0019] Furthermore, the MUSIC super-resolution spectral estimation includes: Construct the autocorrelation covariance matrix for the equivalent free attenuation response corresponding to the actual structural mode frequency band; Singular value decomposition or eigenvalue decomposition is performed on the autocorrelation covariance matrix to obtain the noise subspace; The MUSIC pseudospectrum is constructed by utilizing the orthogonality between the complex exponential steering vector corresponding to the search frequency and the noise subspace; Within a preset frequency search range, the pseudo-spectral peak is searched with a preset search step size to obtain the sling vibration frequency.

[0020] Furthermore, the method also includes: The equivalent free attenuation response corresponding to the actual structural modal frequency band is subjected to Hilbert transform to extract the instantaneous envelope. The high signal-to-noise ratio section at the beginning of the envelope is truncated and logarithmically linearly fitted to estimate the equivalent damping ratio of the suspension cable. The cable force is calculated based on the identified cable vibration frequency, cable length, and mass per unit length.

[0021] Compared with the prior art, the present invention has the following beneficial effects: (1) By using an inner parallelogram region of interest (ROI) and an interpolation-free integer pixel sampling slice, strong background pixels are reduced from participating in the matching, thus avoiding the problem of traditional rectangular ROIs introducing a large number of background regions around the micro-tilt body.

[0022] (2) A logarithmic gradient compression kernel is used to compress the strong background gradient while preserving the edge features of the weak cable body, so that template matching is not easily dominated by the edge of the building window frame or curtain wall.

[0023] (3) By utilizing multi-channel median consensus and search center correction, the global displacement time history can be kept continuous and stable during local reflection, occlusion or single-channel tracking drift.

[0024] (4) Introducing the low-damping vibration characteristics of bridge cables into the frequency identification process transforms the pure mathematical peak search problem into a mode identification problem with physical constraints, which can effectively suppress the low-frequency strong energy interference of UAVs.

[0025] (5) By combining random reduction of the filter bank with MUSIC, the frequency resolution limitation of traditional FFT is broken through under short time window and low frame rate video conditions, and the accuracy of sling frequency recognition is improved.

[0026] (6) The present invention introduces the LOWESS detrending strategy at the front end of the time-frequency domain processing, and uses the estimated fundamental frequency of the sling as a constraint to calculate the smoothing window, so that the detrending cutoff frequency is always in the optimal position between the UAV drift frequency band and the fundamental frequency of the sling; at the same time, the theoretical fundamental frequency interval constraint is introduced to correct the FFT harmonic misjudgment, realizing adaptive preprocessing without manual intervention, and significantly improving the frequency identification robustness under different bridges and different meteorological conditions.

[0027] (7) The present invention further introduces the instantaneous envelope truncation and fitting technique based on Hilbert transform, which can estimate the equivalent damping ratio of the sling under complex disturbances, and provide auxiliary reference for evaluating the effect of the sling damper and judging the fatigue state.

[0028] (8) This invention does not require deep learning training samples or camera intrinsic and extrinsic parameter calibration, making it easy to integrate into UAV automatic inspection systems and bridge digital twin systems. Attached Figure Description

[0029] Figure 1 This invention provides a schematic diagram of the overall process for a method based on UAV vision for eliminating complex backgrounds of bridge suspension cables, robustly extracting micro-vibration displacements, and identifying super-resolution frequencies. Figure 2 This is a schematic diagram illustrating the calibration of the outer rectangular ROI and the inner parallelogram ROI in this invention. Figure 3 This is a schematic diagram of the non-interpolated parallelogram linear slice in this invention, wherein (3a) is a schematic diagram of multiple one-dimensional cross sections, and (3b) is a flowchart of the non-interpolated integer pixel sampling slice. Figure 4 This is a schematic diagram showing the comparison between the one-dimensional gradient characteristics of the suspension cable and logarithmic gradient compression before and after in this invention; Figure 5 This is a schematic diagram of the multi-channel median consensus and abnormal channel search center correction in this invention; Figure 6 This is a schematic diagram of the random reduction processing of the filter bank and the physical coarse screening of the oscillation cycle number in this invention; Figure 7 This is a schematic diagram comparing the traditional Fourier (FFT) spectrum with the MUSIC super-resolution pseudo-spectrum in this invention. Figure 8 This is a schematic diagram of damping identification based on equivalent free decay signal envelope fitting proposed in this invention; Figure 9 This is a schematic diagram of the device in this invention; Figure 10 This is a schematic diagram of the calibration of the double ROI of the sling based on weak perspective conditions in an embodiment of the present invention; Figure 11 This is a comparison chart of the signal purification effects of a fully blind adaptive filter bank and optimized random decrement technique (FB-RDT) in an embodiment of the present invention; Figure 12 This is a comparison diagram of the visual vibration measurement pseudospectrum and the accelerometer reference spectrum of the present invention. Detailed Implementation

[0030] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0031] Example 1 This embodiment provides a method for complex background removal, robust extraction of micro-vibration displacement, and super-resolution frequency recognition of bridge suspension cables based on UAV vision. Figure 1 As shown, it includes the following steps: Acquire video images containing the suspension cables of the bridge under test; In the video image, an outer rectangular region of interest (ROI) and an inner parallelogram region of interest are established. The outer rectangular region of interest is used to define the maximum possible vibration envelope of the sling. The inner parallelogram region of interest is generated based on the line connecting the four corner points of a local segment of the sling after parallelizing the opposite sides, and is located within the outer rectangular region of interest. The region of interest in the inner parallelogram is sampled and sliced ​​without interpolation according to the preset channel step size, resulting in multiple one-dimensional cross sections distributed along the length of the suspension cable. Gradient features are extracted from each one-dimensional cross section and nonlinear logarithmic gradient compression is performed. Based on the compressed gradient features, template matching is performed in parallel on multiple one-dimensional cross sections to obtain displacement estimates for each channel; The displacement estimates of each channel are processed by median consensus, and the initial displacement time history of the sling is obtained by sub-pixel peak interpolation. The initial displacement time history was subjected to LOWESS (Locally Weighted Scatterplot Smoothing) adaptive detrending processing to obtain the true micro-vibration displacement time history of the sling. The actual micro-vibration displacement time history of the sling was processed by the Filter Bank RandomDecrement Technique (FB-RDT) to obtain the equivalent free decay response corresponding to multiple candidate frequency bands; Based on the equivalent free decay response corresponding to each candidate frequency band, the true structural mode frequency band is determined based on the oscillation cycle number physics criterion and energy-purity score; The equivalent free decay response corresponding to the actual structural modal frequency band is subjected to MUSIC (Multiple Signal Classification) super-resolution spectral estimation to obtain the cable vibration frequency.

[0032] The above method comprises two parts: robust tracking in the spatial domain and physical prior frequency identification in the time-frequency domain. It achieves stable extraction of sling micro-vibration displacement and high-precision identification of vibration frequencies without requiring training samples, camera intrinsic and extrinsic parameter calibration, or manual parameter tuning for individual videos. Specifically: In the spatial domain, dual region of interest calibration and interpolation-free parallelogram linear slicing are used to transform the local region of the cable body into multiple one-dimensional horizontal sections, reducing background pixel intrusion from a geometric perspective. At the feature level, a nonlinear logarithmic gradient compression kernel is introduced to reduce the dominant role of strong background gradients in the matching results. At the state estimation level, the consistency of local rigid body motion of the cable is utilized, and single-channel drift caused by local reflections, occlusion, and background edges is suppressed through multi-channel median consensus and abnormal channel search center correction.

[0033] In the time-frequency domain, the initial displacement time history is first preprocessed using LOWESS de-trending, where the smoothing parameter is adaptively determined based on the estimated fundamental frequency, completely removing non-stationary low-frequency drift from the vibration signal. Based on this, to address residual stationary noise and the limitations of a short data window, a joint framework of filter bank stochastic reduction and multiple signal classification (MUSIC) super-resolution spectral estimation is constructed. This framework first extracts equivalent free decay responses within multiple overlapping narrow bands. Then, utilizing the prior knowledge that the low-damped vibration of bridge suspension cables can exhibit multiple complete oscillation periods within a short time window, it coarsely screens the low-frequency non-stationary drift of the UAV. For candidate frequency bands that pass the coarse screening, energy-purity scoring is used to distinguish between the true fundamental frequency, harmonics, and broadband noise. Finally, MUSIC super-resolution spectral estimation is performed on the selected true structural mode frequency bands to obtain the cable vibration frequency.

[0034] Furthermore, the method also includes: performing a Hilbert transform on the equivalent free attenuation response corresponding to the actual structural modal frequency band, extracting the instantaneous envelope, truncating the high signal-to-noise ratio section of the envelope for logarithmic linear fitting, and estimating the equivalent damping ratio of the cable; calculating the cable force based on the identified cable vibration frequency, cable length, and mass per unit length. For the ideal string vibration model, the cable force... T according to calculate, m Mass per unit length L This is the length of the sling. f n For the first n The first natural frequency. For complex cable constraints and cable structures that require consideration of bending stiffness, the cable force can be inverted by combining the identified frequencies with a refined model (e.g., a finite element model).

[0035] If the above methods are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0036] Example 2 This embodiment provides a method for complex background removal, robust extraction of micro-vibration displacement, and super-resolution frequency recognition of bridge suspension cables based on UAV vision. (Reference) Figure 1 As shown, it includes the following steps: 1) The drone hovers near the suspension cables of the bridge under test and collects video of a local section of the suspension cables.

[0037] 2) Establish outer rectangular regions of interest (ROI) and inner parallelogram regions of interest in the video image.

[0038] Specifically, an outer rectangular ROI is determined in the first frame image. This outer rectangular ROI covers the maximum possible vibration envelope of the sling and minimizes the computational area as much as possible. Within the outer rectangular ROI, four corner points near the sling boundary are determined to construct an inner parallelogram ROI, such as... Figure 2As shown. This inner parallelogram must form a tight envelope of the actual physical boundary of the inclined cable, ensuring that the left and right hypotenuses of the parallelogram perfectly match the actual edges of the cable. For a local cable section approximately 1 to 2 meters in length, relative to a UAV observation distance of 2 to 5 meters, the ratio of local depth variation to observation distance is extremely small (typically less than 0.1), and the perspective shrinkage factor remains at a high level of approximately 0.95 to 0.98, fully satisfying the engineering boundary conditions of weak perspective geometry. Therefore, the weak perspective assumption can be adopted to reduce the dimensionality of the complex local perspective mapping to an affine mapping.

[0039] To further verify the robustness of the present invention in complex service environments, Figure 10 The results of ROI calibration under real-world testing conditions are presented. Under conditions with strong interference from urban buildings and river reflections in the background, the outer rectangular global search domain effectively encompasses the maximum vibration range of the suspension cable; the inner parallelogram fine tracking domain performs interpolation-free integer-pixel sampling slices based on weak perspective conditions, precisely fitting the slightly tilted physical boundary of the cable. This dual-track spatial constraint mechanism fundamentally prevents the high-frequency gradient features of the background from contaminating the cable tracking.

[0040] 3) Perform interpolation-free integer pixel sampling slices on the inner parallelogram region of interest according to the preset channel step size to obtain multiple one-dimensional cross sections distributed along the length of the suspension cable.

[0041] like Figure 3 As shown, the non-interpolated integer pixel sampling slice includes: along the left and right boundaries of the region of interest of the inner parallelogram, calculating the horizontal coordinates of the left and right boundaries according to the current cross-sectional vertical coordinates, rounding the calculated sampling coordinates, and reading pixels from the original video image to obtain the one-dimensional cross-section.

[0042] When performing linear slicing on the inner parallelogram ROI, perspective transformation and interpolation resampling of the entire image are not performed. Instead, the sampling start and end positions are calculated linearly along the left and right boundaries of the parallelogram using the ordinate, and the coordinates are rounded before directly reading pixels from the original image. This results in multiple one-dimensional horizontal sections arranged along the length of the suspension cable. This process preserves the original gradient abrupt change information, reduces computational load, and minimizes the impact of background structure on subsequent matching.

[0043] 4) Extract gradient features from each one-dimensional cross section and perform nonlinear logarithmic gradient compression.

[0044] Specifically, the horizontal edge gradient of each one-dimensional cross-section is calculated as the gradient feature. Since the cable cross-section typically exhibits a dark-light-dark or light-dark-light brightness distribution, its first-order gradient contains a stable feature signature composed of negative peaks, zero crossovers, and positive peaks. To suppress strong gradient backgrounds such as building window frames, this embodiment performs nonlinear logarithmic compression on the gradient. A comparison of the cable's one-dimensional gradient features and the logarithmic gradient before and after compression is shown below. Figure 4As shown, logarithmic compression is approximately linear at small signals, preserves weak edges of the suspension cable, and progressively compresses large background gradients without requiring manual setting of steepness or scale parameters.

[0045] Specifically, nonlinear logarithmic gradient compression takes the following form: the original gradient function of the one-dimensional cross-section g ( x ), x Calculate the compression gradient using the x-coordinate of the index. ,in, For symbolic functions, It is the natural logarithm function.

[0046] 5) Perform template matching in parallel on multiple one-dimensional sections.

[0047] In the first frame, a compressed gradient template is extracted. In subsequent frames, the compressed gradient is extracted from the local search region of each channel and normalized cross-correlation template matching is performed to obtain the discrete displacement estimate of each channel.

[0048] 6) Perform median consensus processing on the displacement estimates of each channel, and obtain the initial displacement time history of the sling by sub-pixel peak interpolation.

[0049] like Figure 5 As shown, to avoid the influence of reflections, occlusions, or local background edges on a single channel, this embodiment calculates the median of the displacement estimate for all channels.

[0050] Each one-dimensional cross-section matching channel has an independent search center. The search center is the reference starting pixel coordinate for gradient template matching in the next frame of each channel. The complete matching window is defined with the search center as the reference and a preset search radius. This reduces the amount of image matching calculation and prevents the matching window from continuously shifting to the background area outside the sling.

[0051] If the displacement of a certain channel deviates from the median beyond the consensus tolerance, it is determined to be an abnormal channel and its current displacement is discarded. At the same time, the search center of the next frame of that channel is forcibly corrected to the pixel coordinate position corresponding to the median, thus preventing the continuous accumulation of single-channel tracking errors.

[0052] The normal channel retains the subpixel displacement estimate of the current frame, and superimposes the subpixel displacement estimate of the current frame onto the search center coordinates of the current frame to serve as the search center for the next frame of that channel. The update relationship satisfies: In the formula, The coordinates of the search center in this frame are: d This is a subpixel displacement for this channel, enabling the tracking reference to adaptively move forward following the minute vibrations of the sling.

[0053] This mechanism allows different channels to fail at different times, and as long as the number of healthy channels exceeds half of the total number of channels, the global displacement time history remains stable. It should be noted that the matching template for each channel always uses the fixed template extracted from the first frame and is not updated due to the correction of abnormal channels. This design avoids gradual drift of the template due to local anomalies or changes in illumination, ensuring long-term stability and consistency of tracking.

[0054] Subpixel peak interpolation is achieved using three-point logarithmic parabolic interpolation, and the specific method for obtaining it is as follows: Let the pixel coordinates corresponding to the discrete global maximum peak value in the normalized cross-correlation result be... The corresponding correlation coefficient is The coordinates of the left adjacent pixel are Correlation coefficient The coordinates of the adjacent pixel on the right are Correlation coefficient The natural logarithm of each of the three sets of correlation coefficients was taken, and a quadratic parabola was fitted with pixel position as the independent variable and logarithmic correlation value as the dependent variable. The sub-pixel offset of the parabola's vertex relative to an integer pixel is then calculated. Calculate using the following formula: Final subpixel precise peak coordinates In the formula Integer pixel index, This represents a subpixel-level offset. Subtracting the reference center coordinates of the matching window from the subpixel peak coordinates yields a single-channel subpixel-level displacement estimate; the sequence is then calculated frame by frame and stitched together to obtain the initial displacement time history of the sling.

[0055] The logarithmic parabolic interpolation is based on the assumption that the neighborhood of the cross-correlation peak follows or approximately follows a Gaussian distribution. The logarithmic transformation converts the Gaussian envelope into a quadratic curve, simplifying the solution of extrema and effectively improving the accuracy of displacement measurement.

[0056] Example 3 This embodiment provides a sling frequency super-resolution identification method based on physical priors, referring to... Figure 1 As shown, the initial displacement time history obtained in Example 2 is used as input. Since this time history contains large-amplitude non-stationary UAV low-frequency drift, this example first performs LOWESS detrending processing, where the smoothing parameter is adaptively determined based on the estimated fundamental frequency.

[0057] The adaptive parameter tuning rules are as follows: The amplitude spectrum is obtained by performing a fast Fourier transform on the initial displacement time history, and the highest peak value of the non-zero frequency is searched as a candidate value for the predicted fundamental frequency. To prevent misjudgment due to higher-order harmonic amplitudes of the suspension cable exceeding the fundamental frequency, this method introduces a theoretical fundamental frequency range constraint: the theoretical fundamental frequency range is estimated based on the cable's design length, material, and tension range (for conventional arch bridge or suspension bridge cables, this range can be obtained from design drawings or empirical formulas). (Rough definition). If If the frequency falls within this range, it is taken as the estimated fundamental frequency. ;like If the value is higher than the upper limit of the interval, divide it successively by positive integers greater than 1 until it falls within the interval, and take the corresponding frequency division value as the range. This is to eliminate harmonic interference.

[0058] Then, the equivalent low-pass cutoff frequency LOWESS was set. Preferred coefficient .according to The smoothing parameter is calculated inversely, namely the LOWESS local regression window span. This criterion ensures that drone drift (typically below 0.5Hz) is smoothed into a trend term, while The actual vibration components at or above the specified level are fully preserved.

[0059] As a verification example: for the short suspension cables of the Fuzhou Luozhou Bridge with a fundamental frequency of approximately 4Hz, this criterion automatically matches the LOWESS smoothing parameter. λ Approximately 0.003~0.007; this criterion automatically matches the fundamental frequency of the Jiefang Bridge suspension cables above 6Hz. λ With a value of approximately 0.001, the drift was successfully separated and the fundamental frequency was accurately identified.

[0060] Based on this, since the processed real micro-vibration displacement time history has eliminated low-frequency non-stationary drift, but residual noise still exists, this embodiment continues to perform subsequent filter bank random reduction processing, referring to... Figure 6 As shown, the target frequency band is divided into multiple overlapping narrow bands, for example, a 2Hz narrow band width and a 0.5Hz step size within the range of 0.5 to 15Hz. A random attenuation technique is applied to the filtered signal of each narrow band, with the trigger threshold set to 0.5 times the standard deviation of the narrow band signal. A 3s response segment is extracted and averaged to obtain the equivalent free attenuation response of that narrow band.

[0061] Bridge suspenders are low-damping components, and their free decay response can exhibit multiple complete oscillation cycles within a short time window. In contrast, UAV low-frequency drift typically exhibits lower frequency and stronger non-stationarity, resulting in fewer positive peaks within the same time window. Therefore, this embodiment utilizes the oscillation cycle number criterion to count the number of positive local maxima in the free decay response of each candidate narrowband. Narrowbands with fewer than a preset threshold are discarded. For a 3-second time window and short suspender application scenarios, the threshold can preferably be 5 positive local maxima.

[0062] When multiple candidate narrowbands pass the aforementioned physical coarse screening, this embodiment further calculates the energy-purity score. Specifically, the power spectral density is calculated for each candidate narrowband, and the ratio of the maximum power spectral density within the band to the total power spectral density is defined as the spectral purity. The product of the spectral purity and the total energy is defined as the comprehensive score. Spectral purity is used to suppress broadband noise, and the total energy is used to avoid selecting spurious peaks with excessively weak energy. The candidate narrowband with the highest comprehensive score is selected as the frequency band containing the true structural mode.

[0063] After the aforementioned LOWESS detrending preprocessing and FB-RDT purification, the equivalent free decay response corresponding to the selected real modal frequency band has been completely free of trend terms and colored noise. MUSIC super-resolution spectral estimation is then performed on the equivalent free decay response corresponding to the real structural modal frequency band to obtain the cable vibration frequency.

[0064] Specifically, MUSIC super-resolution spectral estimation includes: constructing an autocorrelation covariance matrix from the equivalent free decay response corresponding to the real structural modal frequency band; performing singular value decomposition or eigenvalue decomposition on the autocorrelation covariance matrix to obtain the noise subspace; constructing a MUSIC pseudospectrum using the orthogonality between the complex exponential steering vector corresponding to the search frequency and the noise subspace; and searching for the pseudospectral peak within a preset frequency search range at a preset search step size to obtain the suspension cable vibration frequency. A comparison between the traditional FFT spectrum and the MUSIC super-resolution pseudospectrum is provided. Figure 7 As shown.

[0065] Based on this, this embodiment can further estimate the equivalent damping ratio of the suspension cable. Specifically, it estimates the equivalent free attenuation response corresponding to the selected real mode frequency band. x ( t Perform Hilbert transform to obtain the analytic signal and extract its instantaneous envelope. A ( t Theoretically, the free decay envelope of a low-damped structure satisfies , where the angular frequency , Modal damping ratio ( n (where the modal order is...) A 0 is a constant. To avoid interference from the low signal-to-noise ratio (SNR) region at the tail end and potential modal beat frequencies, this method truncates the high SNR region at the beginning: the first envelope peak is found as the starting point for fitting, and the fitting endpoint is when the envelope amplitude decays to a preset proportion (e.g., 30%) of that peak. The logarithm of the truncated envelope is taken and a linear fit is performed. The absolute value of the slope of the fitted line is divided by the angular frequency. This is the estimated equivalent damping ratio. In a field test on a short suspension cable, this method successfully fitted an ideal exponentially decaying envelope, such as... Figure 8 As shown, the estimated equivalent damping ratio is 0.381%.

[0066] Example 4 This embodiment provides a robust extraction and super-resolution frequency identification device for micro-vibration displacement of bridge suspension cables under complex backgrounds, such as... Figure 9 As shown, it includes a video acquisition module, an ROI calibration module, a linear slicing module, a gradient compression module, a multi-channel tracking module, a frequency identification module, and a cable force calculation module. The ROI calibration module, the linear slicing module, and the gradient compression module serve as spatial domain processing units, while the multi-channel tracking module serves as a displacement tracking unit. Specifically, the video acquisition module can be implemented by a drone gimbal camera to acquire video images containing the suspension cables of the bridge under test; the ROI calibration module can be implemented by human interaction or a lightweight semantic segmentation network to establish an outer rectangular region of interest and an inner parallelogram region of interest in the video images; the linear slicing module and the gradient compression module perform image preprocessing in the processor. The linear slicing module is used to perform interpolation-free integer pixel sampling slicing on the inner parallelogram region of interest according to a preset channel step size to obtain multiple one-dimensional cross-sections distributed along the length of the suspension cable; the gradient compression module is used to extract gradient features from each one-dimensional cross-section and perform nonlinear logarithmic gradient compression; the multi-channel tracking module is used to perform template matching on multiple one-dimensional cross-sections in parallel based on the compressed gradient features, perform median consensus processing on the displacement estimates of each channel, and obtain and output the initial displacement time history of the suspension cable through sub-pixel peak interpolation; the frequency recognition module performs LOWESS adaptive detrending, filter bank random reduction, and MUSIC super-resolution recognition; the cable force calculation module calculates the cable force based on the cable frequency, length, and mass per unit length.

[0067] The modules mentioned above can be deployed in the edge computing unit of the UAV, or in a ground station, inspection server, or bridge digital twin platform. For automated inspection scenarios, the UAV can periodically collect video of the slings along a fixed route, and the system can automatically output the sling frequency or cable force change trend for bridge operation safety assessment.

[0068] Example 5 This embodiment provides a real-world bridge application of a method for complex background removal, robust micro-vibration displacement extraction, and super-resolution frequency identification of bridge suspension cables based on UAV vision, comprehensively demonstrating the signal purification and adaptive optimization capabilities of this method. The parameter selections for this embodiment are shown in Table 1.

[0069] Table 1 Figure 10 The figure shows a frame of a video captured at 30fps using airborne vision, with a dual ROI calibration map. Figure 11 A comparison of the purification effects of a fully blind adaptive filter bank and an optimized stochastic decrement technique (FB-RDT) is presented. Figure 11The upper part is the original initial displacement signal that is severely polluted by the drone's hovering drift and complex wind noise. This signal is first processed by LOWESS detrending (the smoothing parameter is determined based on the estimated fundamental frequency) to remove the drone's low-frequency drift, and then input into the FB-RDT filter bank. Figure 11 The lower part is the pure equivalent free decay signal extracted after LOWESS pre-cleaning and FB-RDT optimization. This invention abandons the defect of traditional RDT being easily trapped in colored noise interference, and innovatively combines the physical criterion of oscillation cycle number with the energy-purity scoring mechanism: First, by determining the number of oscillation cycles within the time window (for example, requiring it to be greater than a certain threshold, preferably 5 positive local maxima), low-frequency UAV sway artifacts are directly eliminated from the perspective of physical dynamics; then, the product of the local frequency domain purity (Q factor) of each frequency band and the physical absolute energy is calculated as a comprehensive score, and the optimal frequency band of the sling true mode is adaptively locked. Figure 11 The lower half is the pure equivalent free decay signal selected by the highest score of this scoring mechanism. It completely removes colored noise interference, laying the foundation for subsequent high-precision pole search.

[0070] An autocorrelation covariance matrix is ​​constructed for the equivalent free decay response corresponding to the selected frequency band, and the noise subspace is obtained through singular value decomposition. A MUSIC pseudospectrum is constructed using the orthogonality between the complex exponential steering vector and the noise subspace, and a search is performed within a preset frequency range with a search step size of 0.001 Hz. In this embodiment, the maximum number of modes (sources) preset in the MUSIC algorithm is set to 2. The frequency corresponding to the peak value of the pseudospectrum is the vibration frequency of the sling. Compared to the Rayleigh frequency resolution of approximately 0.33 Hz under a 3-second window FFT, this processing significantly improves the frequency estimation accuracy. To quantitatively verify the absolute accuracy of frequency identification in this invention, the visual vibration measurement pseudospectrum of this invention is compared with the reference spectrum of a high-precision contact accelerometer synchronously installed on the surface of the sling. Figure 12 As shown, the contact accelerometer, based on long-term observations, exhibits a reference spectrum (gray dashed area) with a natural physical peak width reflecting the true damping of the structure. In contrast, the visual vibration measurement method proposed in this invention (black solid line), based on FB-RDT purification and MUSIC spatial orthogonality calculation, outputs a super-resolution pseudo-spectrum. This pseudo-spectrum successfully breaks the Rayleigh limit of the traditional Fourier transform, accurately locking the fundamental frequency of the structure with an extremely sharp single peak. Compared to the fundamental frequency identified by a high-precision accelerometer, the relative error of this invention is extremely small (only about 0.04% in this example). This practical verification fully demonstrates that this invention not only achieves non-contact measurement but also, without relying on a large number of long-term continuous observations, its frequency identification accuracy has fully reached the testing standards of traditional professional-grade contact sensors.

[0071] The above embodiments are only used to illustrate the technical concept of the present invention and are not intended to limit the scope of protection of the present invention. The preferred values ​​described above (including but not limited to channel step size, search radius, consensus tolerance, narrowband width, frequency band step size, random decrement time window, trigger threshold, oscillation peak threshold, MUSIC search step size, number of MUSIC sources, etc.) are preferred implementations of the present invention in specific embodiments and do not constitute a limitation on the scope of protection of the present invention. Those skilled in the art can adaptively adjust these values ​​based on actual bridge parameters and video conditions without departing from the technical concept of the present invention. They can also make equivalent substitutions, combinations, or improvements to the ROI calibration method, number of channels, filter bank division, random decrement triggering method, modal scoring function, spectrum estimation algorithm, device deployment method, or cable force conversion model without departing from the spirit and principles of the present invention; such adjustments, equivalent substitutions, combinations, or improvements should all fall within the scope of protection of the present invention.

[0072] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for complex background removal, robust extraction of micro-vibration displacement, and super-resolution frequency recognition of bridge suspension cables based on UAV vision, characterized in that, Includes the following steps: Acquire video images containing the suspension cables of the bridge under test; An outer rectangular region of interest and an inner parallelogram region of interest are established in the video image. The inner parallelogram region of interest is generated based on the line connecting the four corner points of a local segment of the sling, after parallelizing the opposite sides, and is located within the outer rectangular region of interest. The region of interest of the inner parallelogram is sampled and sliced ​​without interpolation according to a preset channel step size to obtain multiple one-dimensional cross sections distributed along the length of the suspension cable. Gradient features are extracted from each one-dimensional cross section and nonlinear logarithmic gradient compression is performed. Based on the compressed gradient features, template matching is performed in parallel on multiple one-dimensional cross sections to obtain displacement estimates for each channel; The displacement estimates of each channel are processed by median consensus, and the initial displacement time history of the sling is obtained by sub-pixel peak interpolation. The initial displacement time history is subjected to LOWESS adaptive detrending processing to obtain the true micro-vibration displacement time history of the sling; The actual micro-vibration displacement time history of the sling is subjected to random reduction processing of the filter bank to obtain the equivalent free attenuation response corresponding to multiple candidate frequency bands; Based on the equivalent free decay response corresponding to each candidate frequency band, the true structural mode frequency band is determined according to the oscillation cycle number physics criterion and energy-purity score; The equivalent free attenuation response corresponding to the actual structural modal frequency band is subjected to MUSIC super-resolution spectral estimation to obtain the cable vibration frequency.

2. The method for complex background removal, robust extraction of micro-vibration displacement, and super-resolution frequency recognition of bridge suspension cables based on UAV vision according to claim 1, characterized in that, The interpolation-free integer pixel sampling slice includes: Along the left and right boundaries of the region of interest of the inner parallelogram, the horizontal coordinates of the left and right boundaries are calculated based on the current cross-sectional ordinate. The calculated sampling coordinates are rounded and pixels are read from the original video image to obtain the one-dimensional cross-section.

3. The method for complex background removal, robust extraction of micro-vibration displacement, and super-resolution frequency recognition of bridge suspension cables based on UAV vision according to claim 1, characterized in that, The template matching includes: Using the compressed gradient features of the first frame of video image as a compressed gradient template, normalized cross-correlation template matching is performed on the compressed gradient features corresponding to the local search region of each channel in subsequent frames to obtain the discrete displacement estimate of each channel.

4. The method for complex background removal, robust extraction of micro-vibration displacement, and super-resolution frequency recognition of bridge suspension cables based on UAV vision according to claim 1, characterized in that, The median consensus processing includes: Each channel is equipped with a search center, which is the starting coordinate of the normalized cross-correlation matching of the next frame for each channel, and is used to limit the local search area for template matching; Calculate the median of the displacement estimates for all channels in the current frame; When the deviation between the displacement estimate of a certain channel and the median exceeds the preset consensus tolerance, the channel is identified as an abnormal channel and its current measurement value is discarded. Force the search center of the next frame of the abnormal channel to be corrected to the position corresponding to the median; For normal channels, retain the subpixel displacement estimate of the current frame, and superimpose the subpixel displacement estimate based on the search center coordinates of the current frame to serve as the search center of the next frame for that channel; Finally, the median of the displacement estimates for all channels in the current frame is used as the global robust displacement output for the current frame of the system.

5. The method for complex background removal, robust extraction of micro-vibration displacement, and super-resolution frequency recognition of bridge suspension cables based on UAV vision according to claim 1, characterized in that, The subpixel peak interpolation is obtained by performing logarithmic parabolic interpolation on the cross-correlation peak points and their left and right adjacent points. The logarithmic parabolic interpolation is based on the assumption that the neighborhood of the cross-correlation peaks follows a Gaussian distribution. It is used to convert the discrete pixel-level displacement estimation into a subpixel-level displacement time history, which serves as the initial displacement time history of the sling.

6. The method for complex background removal, robust extraction of micro-vibration displacement, and super-resolution frequency recognition of bridge suspension cables based on UAV vision according to claim 1, characterized in that, The smoothing parameters of the LOWESS adaptive detrending process are adaptively determined based on the signal characteristics of the initial displacement time history. The determination process includes: The amplitude spectrum is obtained by performing a Fourier transform on the initial displacement time history, and the highest peak value of the non-zero frequency is extracted as the candidate value of the predicted fundamental frequency. ; Preset suspension cable theoretical fundamental frequency range ,like If the frequency falls within the range, it will be used as the estimated fundamental frequency. ;like If the value exceeds the upper limit of the interval, divide it successively by positive integers greater than 1 until the calculation result falls within the theoretical fundamental frequency interval, thus obtaining the estimated fundamental frequency. ; Set equivalent cutoff frequency for 1 / 3 to 1 / 2, according to The smoothing parameters are solved in reverse so that the trend term only includes the low-frequency drift of the UAV and the residual term retains the sling vibration component.

7. The method for complex background removal, robust extraction of micro-vibration displacement, and super-resolution frequency recognition of bridge suspension cables based on UAV vision according to claim 1, characterized in that, The filter bank random decrementing process includes: The target frequency band is divided into multiple overlapping narrow bands. After filtering the signal of each narrow band, a preset multiple of the standard deviation of the narrow band signal is used as the random decrement trigger threshold. The triggering start point is taken only when the narrow band signal crosses the trigger threshold from bottom to top. Response segments of fixed length are continuously intercepted and averaged to obtain the equivalent free attenuation response of the corresponding narrow band.

8. The method for complex background removal, robust extraction of micro-vibration displacement, and super-resolution frequency recognition of bridge suspension cables based on UAV vision according to claim 1, characterized in that, The determination of the true structural modal frequency band includes: The number of positive local maxima in the equivalent free decay response of the candidate frequency band within a preset time window is counted. If the number of positive local maxima is less than a preset threshold, the candidate frequency band is determined to be low-frequency drift or unstructured interference of UAV or other airborne equipment and is removed. Otherwise, the corresponding candidate frequency band is determined to pass the physical criterion of oscillation cycle number. The power spectral density of candidate frequency bands that pass the physical criterion of oscillation cycle number is calculated. The spectral purity is determined based on the ratio of the maximum power spectral density in the frequency band to the total power spectral density of the frequency band. The product of the spectral purity and the absolute energy of the frequency band is used as a comprehensive score. The candidate frequency band with the highest comprehensive score is selected as the true structural mode frequency band.

9. The method for complex background removal, robust extraction of micro-vibration displacement, and super-resolution frequency recognition of bridge suspension cables based on UAV vision according to claim 1, characterized in that, The MUSIC super-resolution spectral estimation includes: Construct the autocorrelation covariance matrix for the equivalent free attenuation response corresponding to the actual structural mode frequency band; Singular value decomposition or eigenvalue decomposition is performed on the autocorrelation covariance matrix to obtain the noise subspace; The MUSIC pseudospectrum is constructed by utilizing the orthogonality between the complex exponential steering vector corresponding to the search frequency and the noise subspace; Within a preset frequency search range, the pseudo-spectral peak is searched with a preset search step size to obtain the sling vibration frequency.

10. The method for complex background removal, robust extraction of micro-vibration displacement, and super-resolution frequency recognition of bridge suspension cables based on UAV vision according to claim 1, characterized in that, The method also includes: The equivalent free attenuation response corresponding to the actual structural modal frequency band is subjected to Hilbert transform to extract the instantaneous envelope. The high signal-to-noise ratio section at the beginning of the envelope is truncated and logarithmically linearly fitted to estimate the equivalent damping ratio of the suspension cable. The cable force is calculated based on the identified cable vibration frequency, cable length, and mass per unit length.

Images