Fuzzy culling and jitter correction method and system for visual monitoring of super high-rise buildings

Abstract

The present invention discloses a method and system for blur removal and jitter correction in visual monitoring of super high-rise buildings, comprising the following steps: a visual sensor acquires a video stream at a fixed frame rate, and locates an initial region of interest (ROI) by mapping prior information; the spatial gradient field is calculated for the ROI image of the current frame, and a Gaussian-weighted two-dimensional gradient structure tensor matrix is ​​constructed; for the clear image sequence that passes blur detection, variational mode decomposition is then used in the time domain to decouple the displacement time series into eigenmode functions of different frequencies, and the low-frequency components are reconstructed to preserve the true deformation of the building; the reconstructed low-frequency displacement signal is output as the final monitoring result. The present invention employs a lightweight algorithm design throughout, significantly reducing the matrix operation dimension of subsequent processing through a dynamic ROI clipping mechanism; the structural tensor eigenvalues ​​are solved using a direct algebraic analytical method, avoiding complex matrix iterative decomposition.

Description

Technical Field

[0001] This invention relates to the field of building structural health monitoring technology, specifically to a method and system for blur removal and jitter correction in visual monitoring of super high-rise buildings. Background Technology

[0002] With the continuous expansion of the construction scale of super high-rise buildings (height ≥ 100m) in my country, structural safety monitoring during construction has become a core aspect of project management. Visual monitoring technology, due to its advantages such as non-contact operation, flexible deployment, and low cost, is widely used in monitoring deformation parameters such as displacement, tilt, and settlement of super high-rise buildings.

[0003] Currently, most mainstream visual monitoring systems employ industrial cameras to capture images at set intervals, using template matching (such as Normalized Cross-Correlation (NCC)) or feature point matching (such as SIFT and ORB) algorithms to track pre-set targets on the building structure and calculate the real-time deformation of the structure. However, in the construction scenarios of super high-rise buildings, visual monitoring equipment is typically installed in areas heavily affected by wind loads and construction machinery, such as climbing formwork platforms and tower crane bodies, leading to two major technical challenges for the system: 1. Image Motion Blur Problem: Mechanical operations such as tower crane lifting and concrete pumping can cause transient strong vibrations in the equipment, resulting in relative displacement between the camera and the target during the exposure time. This leads to severe directional motion blur in the captured image. Directly using blurred images for feature extraction will result in significant mismatches and coordinate jumps, severely contaminating subsequent deformation calculation results. Existing techniques typically use the Laplacian variance method or the Tenengrad gradient function to evaluate image sharpness and remove blurred frames by setting a global threshold. However, these methods are extremely sensitive to ambient lighting and image noise. Under complex outdoor lighting conditions (such as backlighting and low light at night) in high-rise buildings, sharpness scores will fluctuate drastically, causing the fixed global threshold to fail (normal during the day, but misjudged as completely blurred at night). Furthermore, they cannot distinguish between "out-of-focus blur" and "directional motion blur" caused by construction vibrations, resulting in high misjudgment and false negative rates.

[0004] 2. The problem of overlapping camera jitter and structural deformation: The continuous, minute, high-frequency jitter of the installation platform can overlap with the actual low-frequency deformation of the building (such as wind vibration response and slow settlement), resulting in a "spiky" appearance in the calculated displacement curve, which cannot accurately reflect the structural state of the building. Current technologies often smooth the displacement time series directly using moving average filtering or simple low-pass filtering after extracting the target trajectory. These methods are prone to phase delay, and because the vibrations during construction of ultra-high-rise buildings are non-stationary and have a wide bandwidth, simple filtering can easily erase the building's actual "abrupt deformation" or "transient wind vibration response," leading to distortion in deformation monitoring.

[0005] Therefore, there is a need to develop a computationally efficient, illumination-robust, and accurate method for processing visual monitoring data of super high-rise buildings that can distinguish between motion blur and real deformation. Summary of the Invention

[0006] The method and system for blur removal and jitter correction in visual monitoring of super high-rise buildings proposed in this invention can at least solve one of the technical problems in the background art.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A method for blur removal and jitter correction in visual monitoring of super high-rise buildings includes the following steps: S1. The visual sensor acquires video streams at a fixed frame rate, locates the initial region of interest by mapping prior information, tracks the target using a zero-mean normalized cross-correlation algorithm optimized for illumination robustness, and drives the dynamic adaptive update of the region of interest based on the Kalman filter prediction results. S2. Calculate the spatial gradient field for the region of interest image in the current frame, construct a Gaussian weighted two-dimensional gradient structure tensor matrix, determine whether there is directional motion blur caused by construction vibration by analyzing the eigenvalues, and perform interception and removal on the blurred frames. S3. For the clear image sequence that passes the blur detection, firstly, in the spatial domain, the phase correlation method is used to extract the global rigid body translation jitter of the static background area and compensate for the coarse coordinates of the target; then, in the time domain, variational mode decomposition is used to decouple the displacement time series into eigenmode functions of different frequencies and reconstruct the low-frequency components to preserve the true deformation of the building. S4. Output the reconstructed low-frequency displacement signal as the final monitoring result.

[0008] As a preferred embodiment of the blur removal and jitter correction method for visual monitoring of super high-rise buildings according to the present invention, step S1 specifically includes: S11. Using the target's three-dimensional initial coordinates and the camera's intrinsic parameter matrix and extrinsic parameter matrix The initial pixel coordinates of the target on the image plane are calculated using the perspective projection formula, and the initial pixel coordinates are then cropped based on the target's physical dimensions. ; S12. Within the region of interest in the current frame, calculate the zero-mean normalized cross-correlation similarity score between the sliding window and the target standard template, and extract the sub-pixel extreme point where the score is maximum as the precise coordinates of the target. S13. Construct a state vector containing the target position and pixel motion velocity, predict the target position in the next frame using a first-order constant velocity kinematic model, and dynamically expand and shrink the boundary of the region of interest based on the predicted displacement velocity.

[0009] As a preferred embodiment of the blur removal and jitter correction method for visual monitoring of super high-rise buildings according to the present invention, step S2 specifically includes: S21. Convolve the region of interest image with the Sobel or Scharr operator to obtain the horizontal partial derivative of each pixel. and vertical partial derivatives ; S22, in a local window Internally, through a two-dimensional Gaussian function Weighted smoothing of the gradient autocorrelation matrix yields the structure tensor matrix. ; S23. Solving the characteristic equation ,in Since it is an identity matrix, its two eigenvalues ​​can be directly calculated. and , ; S24. Construct the fuzzy evaluation function:

[0010] in, To prevent extremely small positive numbers with a denominator of zero, a preset dynamic threshold is used. ,when Below the dynamic threshold If the frame is blurry, it is determined to be a blurry frame and is removed.

[0011] As a preferred embodiment of the blur removal and jitter correction method for visual monitoring of super high-rise buildings according to the present invention, wherein: the dynamic threshold An adaptive adjustment mechanism is employed: statistical analysis of the most recent N clear images. The mean and standard deviation of the scores will Set as mean minus The threshold is calculated as a multiple of the standard deviation, and is linearly corrected based on the average light intensity within the region of interest.

[0012] As a preferred embodiment of the blur removal and jitter correction method for visual monitoring of super high-rise buildings according to the present invention, wherein: the spatial domain global displacement alignment in step S3 specifically includes: S311. Select a pre-calibrated static background area as a reference window, and perform two-dimensional discrete Fourier transform on the background images of the reference frame and the current frame respectively. S312. Calculate the normalized cross-power spectrum. Eliminate the influence of amplitude spectrum to improve illumination robustness; S313. Perform a two-dimensional discrete Fourier inverse transform on the cross-power spectrum to obtain the phase correlation function. ; S314, Searching The global maximum point is used to obtain the integer pixel offset, and a two-dimensional quadratic surface fitting method is used in the peak neighborhood to extract the sub-pixel offset. ; S315, Adjust the target coarse coordinates Subtract the global offset to obtain the spatially compensated coordinates. .

[0013] As a preferred embodiment of the blur removal and jitter correction method for visual monitoring of super high-rise buildings according to the present invention, the time-domain trajectory signal decoupling in step S3 specifically includes: S321. Input the spatially compensated displacement-time series into the VMD algorithm and set the number of modes. Punishment factor The decomposition yields several eigenmode functions with definite center frequencies; S322. Identify and retain the IMF components with the center frequency within the range of the fundamental frequency of the super high-rise building. S323. Discard the high-frequency IMF components with a center frequency higher than 5Hz, and superimpose the retained low-frequency components to reconstruct the true displacement curve of the building.

[0014] As a preferred embodiment of the blur removal and jitter correction method for visual monitoring of super high-rise buildings described in this invention, the static background area adopts a dynamic update mechanism: every M frames, the ZNCC similarity between the background area and the reference background is recalculated, and when the similarity is lower than a preset threshold, the background of the current frame is updated to a new reference background.

[0015] As a preferred embodiment of the blur removal and jitter correction method for visual monitoring of super high-rise buildings described in this invention, in step S2, after blur frame removal, the missing data is filled in using the prediction result value of Kalman filter. The filled data is marked as low confidence and does not participate in the parameter update of subsequent VMD mode decomposition.

[0016] As a preferred embodiment of the blur removal and jitter correction method for visual monitoring of super high-rise buildings described in this invention, when multiple visual sensors are deployed for collaborative monitoring, steps S1 to S3 are executed on the output of each sensor respectively, and then the displacement data after multi-view correction is fused by weighted average. The weights are positively correlated with the blur removal rate and background similarity of each sensor.

[0017] A blur removal and jitter correction system for visual monitoring of super high-rise buildings includes: The image acquisition module is used to acquire video streams of ultra-tall building targets at a fixed frame rate; The region of interest (ROI) preprocessing module is used to perform image stream capture and preprocessing steps based on the ROI; The fuzz removal module is used to perform directional motion fuzz evaluation and removal steps based on two-dimensional gradient structure tensors; The jitter correction module is used to perform the spatial frequency linkage visual image jitter offset correction steps; The data output module is used to output the actual deformation data of the reconstructed building.

[0018] The beneficial effects of this invention are: This invention employs an anisotropic fuzzy evaluation method based on two-dimensional gradient structure tensor eigenvalue analysis, which can specifically identify directional motion fuzziness caused by construction vibration. Unlike traditional global variance calculation, this method effectively overcomes the impact of illumination changes and image noise on sharpness evaluation by constructing a fuzzy evaluation function and combining it with a dynamic adaptive threshold adjustment mechanism. This invention proposes a spatial-frequency linkage jitter correction architecture of "spatial domain global alignment + temporal domain mode decomposition". In the spatial domain, the phase correlation method is used to extract the global rigid body jitter of the camera, so as to achieve sub-pixel level instantaneous jitter compensation. This invention employs a lightweight algorithm design throughout, significantly reducing the dimensionality of matrix operations in subsequent processing through a dynamic ROI pruning mechanism; the eigenvalues ​​of the structural tensor are solved using a direct algebraic analytical method, avoiding complex matrix iterative decomposition. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the steps of the method and system for blur removal and jitter correction in visual monitoring of super high-rise buildings according to the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0021] like Figure 1 As shown, the method for blur removal and jitter correction in visual monitoring of super high-rise buildings includes the following steps: S1. The visual sensor acquires video streams at a fixed frame rate, locates the initial region of interest by mapping prior information, tracks the target using a zero-mean normalized cross-correlation algorithm optimized for illumination robustness, and drives the dynamic adaptive update of the region of interest based on the Kalman filter prediction results. Specifically, to meet the extremely low computing power limitations of edge computing terminals, a region of interest (ROI) pruning mechanism based on spatial mapping and dynamic tracking is adopted. The specific implementation includes the following steps: S11. Initial ROI mapping and localization based on prior information: During the initial initialization or reset of the system, the initial pixel coordinates of the target on the image plane are directly calculated using the perspective projection model of the camera by utilizing the prior calibration information of the physical space. Assume that the prior information includes: the absolute three-dimensional initial coordinates of the target in the confined space or architectural model. The camera's intrinsic parameter matrix K (including the effective focal length) and principal point coordinates ), and the camera's extrinsic parameters relative to the world coordinate system (rotation matrix R and translation vector T).

[0022] The initial target center pixel position in the image coordinate system The following perspective projection formula can be used to solve the problem:

[0023] in, The depth value in the camera coordinate system, with ( , Centered on the target, and combining the target's physical size and prior depth, a certain amount of redundant pixel width W and height H are extended outwards to crop out the initial target. This method avoids blind full-graph search during initial startup; S12. Considering the drastic changes in outdoor lighting at super high-rise buildings (such as backlighting and shadows) and the limited computing power of edge computing, this invention adopts the zero-mean normalized cross-correlation (ZNCC) algorithm optimized for lighting robustness as the core tracking logic. In the Within the frame, extract the pre-stored target standard template. Calculate the sliding window Similarity score to template The ZNCC calculation formula is as follows:

[0024] in, This represents the mean value of a local image within the current search window. This is the template mean.

[0025] Because ZNCC subtracts the mean and divides by the variance, the system fundamentally eliminates the linear changes in overall image brightness and contrast caused by changes in the angle of sunlight and cloud cover due to high-rise buildings at the mathematical level, ensuring extremely high robustness of target detection; the sub-pixel extremum point where the score S(u,v) is maximized is extracted, which is the precise coordinate of the target in the current frame. ; S13. Super high-rise buildings will experience large-scale sudden displacements during strong wind loads or when tower cranes lift them. If the ROI window position is fixed or only depends on the position of the previous frame, the target is very likely to "jump out" of the ROI boundary, resulting in tracking loss. To address this, this invention introduces a first-order constant velocity kinematic Kalman filter to drive the dynamic update of the ROI.

[0026] Construct the target's state vector in the image pixel plane (Including position and pixel motion speed). Utilizing the previous frame (k 1) The optimal estimated state is obtained by predicting the possible center position of the target in the k-th frame using the state transition matrix A. :

[0027] in, The time interval between two frames; Dynamic scaling boundary logic: the k-th frame The cut center is forced to be set to the predicted coordinates. Simultaneously, the system monitors the state error covariance matrix in real time. trace or displacement velocity .

[0028] When the speed increases (a large-amplitude shaking of the building is detected), the system dynamically enlarges the width (W) and height (H) of the ROI proportionally to accommodate larger displacement changes that may occur in the next frame. When the speed decreases (the building is in a stable state), the system automatically tightens the boundaries of the ROI, removes redundant pixels, and minimizes the matrix operation dimension of subsequent blur removal and high-frequency jitter correction.

[0029] S2. Calculate the spatial gradient field for the region of interest image in the current frame, construct a Gaussian weighted two-dimensional gradient structure tensor matrix, determine whether there is directional motion blur caused by construction vibration by analyzing the eigenvalues, and perform interception and removal on the blurred frames. Specifically, this step constructs a fuzzy evaluation mechanism based on the analysis of two-dimensional gradient structure tensors and their eigenvalues. The specific implementation process is as follows: S21. Image spatial gradient field calculation: Take the region of interest image of the current k-th frame output in step one. .

[0030] First, the image is convolved using a preset discrete differential operator (such as the Sobel operator or the Scharr operator) to calculate the horizontal partial derivative of each pixel (x, y) within the ROI. and vertical partial derivatives :

[0031] S22. Construction of the two-dimensional gradient structure tensor matrix: In order to extract the texture orientation structure features in the local neighborhood and enhance the algorithm's ability to resist additive noise in the image, the autocorrelation matrix of the gradient image is locally Gaussian smoothed and weighted. Define a local window (e.g., 5×5 or 7×7 pixels) and a two-dimensional Gaussian weighted function The smoothed two-dimensional gradient structure tensor matrix is ​​calculated. :

[0032] because The tensor matrix S of this structure is a real symmetric matrix, that is... .

[0033] S23. Algebraic Analysis and Directional Motion Analysis of Eigenvalues ​​of Structure Tensor: Two eigenvalues ​​of the structure tensor matrix S and This implies the energy distribution pattern of gray-level changes within local image regions. To achieve efficient solution on edge computing terminals and avoid complex matrix iterative decomposition, this system adopts an algebraic analytical method that directly solves the characteristic equation.

[0034] Let the characteristic equation of matrix S be: (where I is the identity matrix), after expansion, we get a quadratic equation in one variable:

[0035] Note that the coefficients of the equation are the trace (Tr) and determinant (Det) of matrix S:

[0036]

[0037] The two eigenvalues ​​can be directly solved using the quadratic formula. and (and agreed) ≥ ):

[0038]

[0039] Eigenvalue-based physical state mapping logic: Smooth region: If =0 and =0 indicates that the image has no texture (such as a solid-color sky background) and has no tracking value.

[0040] Clear target (corner points / strong texture): If and Both are relatively large, and the ratios are... / The proximity indicates that the region exhibits strong grayscale variations in all directions, representing a high-quality target feature that has not become blurred.

[0041] Vibration-induced directional blurring: If construction vibrations cause one-dimensional relative displacement of the target, the image features will be stretched into edges. In this case, the gradient along the direction perpendicular to the edge (corresponding to...) The gradient is still relatively large, but the gradient parallel to the direction of the motion trail will decay sharply, leading to... →0. That is... At that time, it was determined that severe vibration and motion blur had occurred.

[0042] S24. Adaptive Sharpness Assessment and Removal Decision: To quantify the above physical state, the system constructs an anisotropic fuzzy evaluation function:

[0043] in To prevent extremely small positive numbers with a denominator of zero. When vibration ambiguity occurs ( (Extremely small), the molecule approaches zero, and the score is... A sudden drop.

[0044] System preset dynamic threshold When the ROI evaluation score of the current frame < At that time, the system directly determines that the frame is affected by "construction vibration pollution", performs physical interception and discards it, and prevents the erroneous feature coordinates from entering the subsequent time series calculation.

[0045] S3. For the clear image sequence that passes the blur detection, firstly, in the spatial domain, the phase correlation method is used to extract the global rigid body translation jitter of the static background area and compensate for the coarse coordinates of the target; then, in the time domain, variational mode decomposition is used to decouple the displacement time series into eigenmode functions of different frequencies and reconstruct the low-frequency components to preserve the true deformation of the building. For the clear image sequence that passed step S2, perform jitter correction: S311. Spatial Domain Global Displacement Alignment and Rigid Body Jitter Elimination Based on Phase Correlation Method: A "static background region" (such as a distant, stable building or a rigid feature on the same substrate) that does not deform with the target image is selected as the reference window. This region is assumed to be M×N pixels in size. The background image of the reference frame (such as the first or previous frame) is defined as... The background image of the k-th frame where the vibration shift occurs is... Assume the camera experiences pure translational shaking ( , If both satisfy the following conditions in the spatial domain:

[0046] To accurately extract this displacement, the system performs the following frequency domain calculation procedure: Two-dimensional Discrete Fourier Transform (2D-DFT): Transform the background images of the reference frame and the current frame from the spatial domain to the frequency domain, respectively. and They are respectively and Two-dimensional discrete Fourier transform:

[0047]

[0048] Based on the translation properties of the Fourier transform, the relationship between the two in the frequency domain can be expressed as:

[0049] S312. Calculate the cross-power spectrum: Extract the phase difference matrix between the two, i.e., calculate the normalized cross-power spectrum C(u,v). This step desensitizes the algorithm to changes in illumination by eliminating the amplitude spectrum.

[0050] in, represent The complex conjugate of , |·| represents the modulo operation.

[0051] S313. Inverse Fourier Transform and Correlation Pulse Extraction: cross power spectrum Performing a two-dimensional inverse discrete Fourier transform (2D-IDFT) yields the phase correlation function in the spatial domain. :

[0052] In an ideal situation, It is a Dirac impulse function that is localized everywhere else in the image matrix, except at coordinates ( , A sharp energy peak appears at ().

[0053] S314, Subpixel-level offset extraction and coordinate compensation: By finding relevant functions By finding the global maximum point, we can obtain the camera translation offset in integer pixels:

[0054] Optimized features: To meet the high-precision requirements of monitoring minute deformations in ultra-high-rise buildings, the system further optimizes the relevant peak values ​​( , Within the neighborhood, sub-pixel-level extreme coordinates are extracted using a two-dimensional quadratic surface fitting method.

[0055] Finally, the coarse coordinates of the target object extracted in the current frame are... , Subtracting the calculated global rigid body translation, we obtain the spatially compensated coordinates after eliminating camera jitter. :

[0056] After global alignment in this spatial domain, the output coordinate sequence is used to proceed to the next stage of temporal mode decomposition (VMD).

[0057] Furthermore, S32, time-domain trajectory signal decoupling (VMD adaptive mode extraction): Even after spatial alignment, the displacement time series (x(t), y(t)) of the target still inevitably contains high-frequency structural vibration noise. This one-dimensional time series is then introduced into the variational mode decomposition (VMD) algorithm.

[0058] VMD can adaptively decompose a mixed displacement signal into K intrinsic mode functions (IMFs) with defined center frequencies. In this invention, the number of modes K=4, the penalty factor α=3000, and the noise margin τ=0 are set to obtain 4 IMF components.

[0059] Based on the unique low-frequency properties of super high-rise buildings (the fundamental frequency is usually between 0.1Hz and 1Hz), the IMF components with a center frequency in the range of 0.05Hz to 1.5Hz are identified and retained; the high-frequency IMF components with a center frequency higher than 5Hz (corresponding to construction machinery vibration noise) are discarded, and the retained low-frequency components are superimposed and reconstructed to obtain the actual displacement curve of the building.

[0060] S4. Output the reconstructed low-frequency displacement signal as the final monitoring result; Specifically, the real deformation data is output: the low-frequency displacement signal reconstructed in step S3 is output as the final result; this result not only eliminates the instantaneous jump offset of the camera, but also retains the real wind vibration response and slow settlement / tilt deformation of the building, realizing robust monitoring of the deformation of super high-rise building structures.

[0061] To further verify the actual technical effectiveness of this invention in the extremely harsh environment of super high-rise building construction, the following detailed description is provided in conjunction with specific engineering embodiments, comparative experimental data, and displacement curves: 1. Experimental platform setup and parameter settings The visual monitoring system of this invention was deployed at the construction site of the core tube of a 450-meter super high-rise building under construction (elevation approximately 380 meters).

[0062] Hardware Deployment: High-resolution visual sensors are mounted on a stable adjacent building 200 meters from the core tube to capture video streams at a fixed frame rate of 60fps. The monitoring target uses an 800mm×800mm LED self-illuminating feature plate, with an initial pixel equivalent of 1.5mm / pixel after calibration.

[0063] Algorithm parameter settings: Configure according to steps S1 to S3 of this invention. Specifically, in step S2, the local window for extracting structural tensor feature values ​​is set to 7×7 pixels; in step S3, the number of modes of the VMD algorithm is set to K=4, and the penalty factor α=3000.

[0064] Experimental control group setup: Existing technology group (comparison group A): adopts the traditional "global sharpness assessment based on Laplacian variance + standard NCC template matching and tracking + moving average filtering algorithm with window length N=50".

[0065] The invention group (comparative group B) adopts "directional fuzz removal based on two-dimensional gradient structure tensor + ZNCC tracking + spatial frequency linkage jitter correction based on VMD".

[0066] 2. Typical operating conditions and quantitative comparison data The experiment extracted three typical working conditions that are the most challenging in the construction of super high-rise buildings, and conducted a quantitative comparison after continuous operation for 72 hours. The results are as follows: Operating Condition 1: Low-light environment at night (testing light resistance) Scene description: Nighttime with no main lighting, the overall image is dark and accompanied by Gaussian noise.

[0067] Data comparison: In comparison group A, the false rejection rate of sharp frames was as high as 68.5%, and the target tracking success rate was only 42.1%; the present invention group (comparison group B), relying on the mean normalization characteristics and structural tensor analysis of the ZNCC algorithm, reduced the false rejection rate of sharp frames to 3.2%, while maintaining a tracking success rate of 98.7%. The present invention effectively overcomes the impact of illumination changes and image noise on sharpness assessment.

[0068] Operating Condition 2: Transient Strong Vibration During Tower Crane Lifting (Testing Orientation Fuzzy Recognition and High-Frequency Filtering) Scene description: A large luffing jib crane is lifting a fully loaded machine, causing the camera bracket to experience severe high-frequency mechanical vibrations of 15Hz~25Hz, resulting in severe one-way motion blur in the image.

[0069] Data comparison: Due to the inability to distinguish between "out-of-focus" and "motion blur", the false negative rate for directional blur in comparison group A reached 45.3%, and the mean square error (RMSE) for displacement measurement reached 14.8mm. This invention reduces the false negative rate for directional blur to 1.5%, and uses the phase correlation method in the spatial domain to extract the global rigid body jitter of the camera, achieving sub-pixel level instantaneous jitter compensation, with a displacement mean square error of only 1.2mm.

[0070] Operating Condition 3: Transient wind vibration under strong gusts (testing the fidelity of low-frequency true displacement) Scene description: Affected by high-altitude gusts of wind, the building experiences low-frequency realistic swaying with a frequency of approximately 0.3Hz and an amplitude of approximately 40mm.

[0071] Data comparison: The system phase delay of comparison group A is as high as 650ms, and the real peak is distorted by the sliding filter error; the present invention group (comparison group B) decouples the displacement time series into eigenmode functions of different frequencies by using variational mode decomposition in the time domain, perfectly separating high-frequency mechanical noise from low-frequency wind vibration, with a system phase delay of <20ms and a real wind vibration waveform overlap of 96%.

[0072] Conclusion: The above embodiments and experimental data fully demonstrate that the present invention adopts a lightweight algorithm design throughout the process, successfully solving the two major problems of "image motion blur" and "camera shake and structural deformation aliasing" in the construction of super high-rise buildings. The monitoring accuracy, system robustness and real-time performance under harsh working conditions have been improved.

[0073] In addition, the present invention also provides a blur removal and jitter correction system for visual monitoring of super high-rise buildings, including: The image acquisition module is used to acquire video streams of ultra-tall building targets at a fixed frame rate; The region of interest (ROI) preprocessing module is used to perform image stream capture and preprocessing steps based on the ROI; The fuzz removal module is used to perform directional motion fuzz evaluation and removal steps based on two-dimensional gradient structure tensors; The jitter correction module is used to perform the spatial frequency linkage visual image jitter offset correction steps; The data output module is used to output the actual deformation data of the reconstructed building.

[0074] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above method.

[0075] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0076] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for fuzzy removal and jitter correction of visual monitoring of super high-rise buildings, characterized in that, Includes the following steps: S1. The visual sensor acquires video streams at a fixed frame rate, locates the initial region of interest by mapping prior information, tracks the target using a zero-mean normalized cross-correlation algorithm optimized for illumination robustness, and drives the dynamic adaptive update of the region of interest based on the Kalman filter prediction results. S2. Calculate the spatial gradient field for the region of interest image in the current frame, construct a Gaussian weighted two-dimensional gradient structure tensor matrix, determine whether there is directional motion blur caused by construction vibration by analyzing the eigenvalues, and perform interception and removal on the blurred frames. S3. For the clear image sequence that passes the blur detection, firstly, in the spatial domain, the phase correlation method is used to extract the global rigid body translation jitter of the static background area and compensate for the coarse coordinates of the target; then, in the time domain, variational mode decomposition is used to decouple the displacement time series into eigenmode functions of different frequencies and reconstruct the low-frequency components to preserve the true deformation of the building. S4. Output the reconstructed low-frequency displacement signal as the final monitoring result.

2. The method of claim 1, wherein the method further comprises: Step S1 specifically includes: S11, using target three-dimensional initial coordinates, camera intrinsic matrix and extrinsic matrix , through the perspective projection formula to solve the initial pixel coordinates of the target in the image plane, combined with the target physical size to clip the initial ; S12. Within the region of interest in the current frame, calculate the zero-mean normalized cross-correlation similarity score between the sliding window and the target standard template, and extract the sub-pixel extreme point where the score is maximum as the precise coordinates of the target. S13. Construct a state vector containing the target position and pixel motion velocity, predict the target position in the next frame using a first-order constant velocity kinematic model, and dynamically expand and shrink the boundary of the region of interest based on the predicted displacement velocity.

3. The method of claim 1, wherein the method further comprises: determining a blur distance of the camera; and determining a blur distance of the camera based on the blur distance of the camera and the height of the camera. Step S2 specifically includes: S21, using Sobel operator or Scharr operator and the image of the region of interest convolution, get each pixel horizontal partial derivative and vertical partial derivative ; S22、in the local window by a two-dimensional Gaussian function The gradient autocorrelation matrix is weighted and smoothed to obtain a structure tensor matrix ; S23, by solving the characteristic equation where is the identity matrix, the two eigenvalues are directly calculated and , ; S24. Construct the fuzzy evaluation function: in, To prevent extremely small positive numbers with a denominator of zero, a preset dynamic threshold is used. ,when Below the dynamic threshold If the frame is blurry, it is determined to be a blurry frame and is removed.

4. The method for blur removal and jitter correction in visual monitoring of super high-rise buildings according to claim 3, characterized in that: The dynamic threshold An adaptive adjustment mechanism is employed: statistical analysis of the most recent N clear images. The mean and standard deviation of the scores will Set as mean minus The threshold is calculated as a multiple of the standard deviation, and is linearly corrected based on the average light intensity within the region of interest.

5. The method for blur removal and jitter correction in visual monitoring of super high-rise buildings according to claim 1, characterized in that: The spatial domain global displacement alignment in step S3 specifically includes: S311. Select a pre-calibrated static background area as a reference window, and perform two-dimensional discrete Fourier transform on the background images of the reference frame and the current frame respectively. S312. Calculate the normalized cross-power spectrum. Eliminate the influence of amplitude spectrum to improve illumination robustness; S313. Perform a two-dimensional discrete Fourier inverse transform on the cross-power spectrum to obtain the phase correlation function. ; S314, Searching The global maximum point is used to obtain the integer pixel offset, and a two-dimensional quadratic surface fitting method is used in the peak neighborhood to extract the sub-pixel offset. ; S315, Adjust the target coarse coordinates Subtract the global offset to obtain the spatially compensated coordinates. .

6. The method for blur removal and jitter correction in visual monitoring of super high-rise buildings according to claim 5, characterized in that: The time-domain trajectory signal decoupling in step S3 specifically includes: S321. Input the spatially compensated displacement-time series into the variational mode decomposition algorithm and set the number of modes. Punishment factor The decomposition yields several eigenmode functions with definite center frequencies; S322. Based on the fundamental frequency range of super high-rise buildings, identify and retain the intrinsic mode function components with the center frequency within that range; S323. Discard the high-frequency intrinsic mode function components with a center frequency higher than 5Hz, and superimpose the retained low-frequency components to reconstruct the true displacement curve of the building.

7. The method for blur removal and jitter correction in visual monitoring of super high-rise buildings according to claim 6, characterized in that: The static background region adopts a dynamic update mechanism: every M frames, the zero-mean normalized cross-correlation similarity between the background region and the reference background is recalculated. When the similarity is lower than a preset threshold, the background of the current frame is updated to the new reference background.

8. The method for blur removal and jitter correction in visual monitoring of super high-rise buildings according to claim 6, characterized in that: After the blurry frames are removed in step S2, the missing data is filled in using the prediction results of Kalman filtering. The filled data is marked as low confidence and does not participate in the parameter update of subsequent VMD mode decomposition.

9. The method for blur removal and jitter correction in visual monitoring of super high-rise buildings according to claim 1, characterized in that: When multiple vision sensors are deployed for collaborative monitoring, steps S1 to S3 are executed on the output of each sensor. Then, the displacement data after multi-view correction is fused by weighted averaging. The weights are positively correlated with the blur removal rate and background similarity of each sensor.

10. A fuzz removal and jitter correction system for visual monitoring of super high-rise buildings, implementing the fuzz removal and jitter correction method for visual monitoring of super high-rise buildings as described in any one of claims 1-9, characterized in that, include: The image acquisition module is used to acquire video streams of ultra-tall building targets at a fixed frame rate; The region of interest (ROI) preprocessing module is used to perform image stream capture and preprocessing steps based on the ROI; The fuzz removal module is used to perform directional motion fuzz evaluation and removal steps based on two-dimensional gradient structure tensors; The jitter correction module is used to perform the spatial frequency linkage visual image jitter offset correction steps; The data output module is used to output the actual deformation data of the reconstructed building.

Images