A method for identifying the vibration modal and displacement of a steel tower by video tracking

Abstract

The application discloses a kind of steel tower vibration modal and displacement identification method of video tracking, it is related to engineering asset operation and maintenance management technical field, including the following steps: scale calibration step, based on the calibration parameter of monitoring station and camera, calculate scale calibration coefficient, provide uniform quantitative benchmark for whole link calibration;Input layer scale calibration step, mark point topology optimization layout step, space-time frequency domain interference decoupling step, multi-view heterogeneous feature federal fusion step, low-texture area vibration state reasoning step, micro-vibration signal modal identification step;Closed-loop monitoring control step, incremental model updating and data management step, by dynamically adjusting attention layer receptive field, ensure the accuracy of interference decoupling under different distances, effectively separate camera shake, wind-induced interference and steel tower itself vibration component, solve the problem that micro-vibration information is submerged;At the same time, through scale normalization and dynamic weight adjustment, realize the effective federal fusion of multi-view heterogeneous data.

Description

Technical Field

[0001] This invention belongs to the field of engineering asset operation and maintenance management technology, specifically a video tracking method for identifying the vibration modes and displacement of steel towers. Background Technology

[0002] As core infrastructure in fields such as power transmission and communications, the structural health of towering steel towers directly affects the operational safety of these infrastructures. During long-term service, steel towers are susceptible to structural damage due to factors such as wind loads and environmental corrosion, which can lead to safety accidents such as collapse. Therefore, long-term vibration and displacement monitoring of steel towers is a core means of ensuring their safe operation.

[0003] Traditional steel tower monitoring often employs contact-based sensing solutions, collecting structural response data by deploying sensors such as accelerometers and strain gauges within the tower. However, this approach suffers from drawbacks such as complex wiring, high maintenance costs, and limited coverage, making it unsuitable for the full lifecycle monitoring needs of large-span, tall steel towers. In recent years, machine vision monitoring technology, with its advantages of being non-contact, low-cost, and having wide coverage, has gradually become the mainstream solution for steel tower health monitoring. However, for long-distance visual monitoring scenarios of tall steel towers, existing technologies still face many unresolved technical challenges: Existing visual monitoring models are mostly designed for short-distance scenarios. When the monitoring distance exceeds 100 meters, the camera's own tiny shakes will be amplified by long-distance projection, causing the micro-displacement measurement error to increase exponentially. At the same time, the actual spatial scale corresponding to the camera pixels varies significantly at different monitoring distances, causing a mismatch across the entire chain of input data scale, model receptive field scale, feature extraction scale, state inference scale, and model update scale. This leads to a sharp drop in the accuracy of general machine learning models in micro-vibration recognition in long-distance scenarios. Summary of the Invention

[0004] The purpose of this invention is to propose a video-tracking method for identifying the vibration modes and displacements of steel towers, comprising the following steps: The scale calibration step calculates the scale calibration coefficient based on the calibration parameters of the monitoring station and camera, providing a unified quantitative benchmark for end-to-end calibration; The input layer scale calibration step involves resampling the original video frames according to the scale calibration coefficients to unify the spatial scale of the input data and provide standardized input for subsequent processing. The topology optimization layout step for marker points involves extracting topological location features, visual visibility features, and vibration monitoring contribution features of candidate marker points based on a calibrated unified input scale. The features are then weighted and summed to obtain a priority score. Candidate points are sorted from high to low according to the priority score, and the point layout density is adaptively adjusted to provide an effective marking basis for feature extraction. The spatiotemporal frequency domain interference decoupling step dynamically adjusts the receptive field parameters of the attention layer according to the scale calibration coefficient, performs interference decoupling processing on the collected raw monitoring data, removes camera shake and wind-induced environmental interference components, and outputs purified steel tower motion time series data, providing a purified data source for subsequent feature fusion. The multi-view heterogeneous feature federation step involves each edge monitoring node normalizing the scale of its local features based on the local scale calibration coefficient, uploading the normalized local convolutional neural network model parameters to the central node, calculating the fusion weights based on the scale calibration coefficients of each node, and performing a weighted average of the model parameters of each node to obtain the global model parameters, thus completing federated learning and outputting a globally unified motion feature set. The vibration state inference step for low-texture regions involves using the fused feature set and the normalized stiffness constraints of the steel tower members to complete the missing motion features of the low-texture regions based on a knowledge-enhanced graph neural network. This outputs a standardized vibration state feature vector, providing complete state features for subsequent modal identification. The micro-vibration signal modal identification step, based on the state feature vector and a lightweight Transformer model with subspace projection and quantization compression, completes the high-precision identification of the steel tower vibration modal parameters and the entire tower displacement field, and outputs the real-time identification results at the edge end, providing a decision basis for subsequent closed-loop control; The closed-loop monitoring and control steps adaptively adjust the monitoring sampling parameters and the monitoring weights of marked points based on the modality recognition results, optimize the data acquisition quality and monitoring resource allocation, and trigger the model update command based on meta-learning. The incremental model update and data management steps involve iteratively updating the model parameters based on the meta-learning continuous incremental learning algorithm according to the model update instructions, while storing the monitoring data in a hierarchical and classified manner.

[0005] Furthermore, the scale calibration step specifically includes: First, the camera's extrinsic parameter calibration process is executed. A standard checkerboard calibration plate is set up on the surface of the steel tower, and the collected calibration plate image is subjected to sub-pixel level corner detection. Then, based on epipolar geometric constraints, pose calculation is performed to obtain the actual spatial distance from the monitoring station to the surface of the steel tower. Then, the camera's intrinsic parameter calibration process is executed. By fully calibrating the radial and tangential distortion of the camera lens, the error caused by lens distortion is eliminated, and the actual focal length parameter of the camera lens is obtained. Finally, based on the obtained actual distance and focal length parameters, the scale calibration coefficient is calculated.

[0006] Furthermore, the input layer scale calibration step specifically includes: Based on the calculated scale calibration coefficients, the original input video frames are resampled using a bilinear interpolation algorithm. The resampling ratio is adaptively adjusted according to the scale calibration coefficients so that the actual spatial scale of the input frames at different monitoring distances is unified to the reference scale.

[0007] Furthermore, the topology optimization deployment step of the marker points specifically includes: extracting the topological location features, visual visibility features, and vibration monitoring contribution features of candidate marker points based on the calibrated unified input scale; obtaining a priority score by weighted summation of the features; sorting the candidate points from high to low according to the priority score; for long-distance areas with larger scale calibration coefficients, the deployment density of marker points is higher; by adjusting the density, the number of marker points in a unit of actual space is made uniform; during the deployment process, candidate points are screened to exclude invalid points that are occluded or cannot be observed by the camera, ensuring that all marker points can be effectively observed, thus completing the layered deployment of marker points.

[0008] Furthermore, the spatiotemporal frequency domain interference decoupling step specifically includes: adjusting the receptive field parameters of the attention layer according to the scale calibration coefficient, the size of the receptive field being adjusted according to the scale calibration coefficient so that the actual spatial scale corresponding to the attention receptive field at different distances is unified as the reference scale; taking the synchronously acquired camera motion data as input, the spatial features of the video frame and the frequency domain features of the motion sequence are weighted through the adjusted attention layer; in the spatial domain, the interference area caused by camera shake is located; in the frequency domain, the low-frequency wind-induced interference and the high-frequency vibration component of the steel tower itself are distinguished, and the camera shake component, the wind-induced interference component, and the vibration response component of the steel tower itself are separated to complete the decoupling of the interference signal and obtain the steel tower motion time sequence data.

[0009] Furthermore, the multi-view heterogeneous feature federated fusion step specifically includes: each edge camera node performs scale normalization processing on the locally extracted motion features based on the local scale calibration coefficient, unifying the scale of the local features to a reference scale, eliminating scale differences between different nodes, and ensuring that the features of different nodes have a unified scale reference; the normalized local convolutional neural network model parameters are uploaded to the central node; the central node calculates the fusion weight of each node based on the scale calibration coefficient of each camera, with longer-distance nodes having higher fusion weights for larger scale calibration coefficients; and the model parameters of each node are weighted and averaged to obtain the global model parameters.

[0010] Furthermore, the vibration state inference step for the low-texture region specifically includes: normalizing the stiffness matrix of the steel tower members according to the scale calibration coefficient, unifying the scale of the stiffness constraints to the reference scale, and eliminating scale differences in the stiffness constraints; integrating the normalized stiffness constraints into the message passing process of the graph neural network to construct a knowledge-enhanced graph neural network with stiffness constraints, and integrating the prior knowledge of the structure into the model's inference process; in the message passing process of the graph neural network, completing the missing motion features of the low-texture region according to the stiffness constraints, and completing the motion features of these low-texture regions based on the motion data of adjacent nodes using the prior knowledge of the stiffness constraints; and outputting the vibration state feature vector.

[0011] Furthermore, the micro-vibration signal modal identification step specifically includes: constructing a time series matrix and performing singular value decomposition on the motion time series data of the marked points using a random subspace identification algorithm, completing subspace projection, and extracting modal feature embeddings to map the original high-dimensional time series data to a low-dimensional modal feature space to eliminate the influence of noise; inputting the feature embeddings into the quantized Transformer model, and using divergence to calibrate the quantization error during the quantization process to ensure that the accuracy loss of the quantized model at different distances is controlled within an acceptable range; and using this model, completing the identification of the modal parameters of the steel tower and the displacement field of the entire tower.

[0012] Furthermore, the closed-loop monitoring and control steps specifically include: acquiring the highest frequency component of the identified vibration signal, adjusting the sampling frame rate to ensure it is always more than twice that of the highest frequency component, simultaneously adjusting the exposure time according to the adjustment of the sampling frame rate to ensure stable image brightness, and correcting the monitoring weight of the marker points in the corresponding areas according to the identified vibration amplitude distribution. For areas with higher amplitude, more monitoring resources are allocated to improve the monitoring accuracy of those areas, while for areas with lower amplitude, the monitoring weight is reduced to save resources, thus achieving the allocation of monitoring resources. The monitoring parameters are adjusted according to the real-time signal characteristics.

[0013] Furthermore, the incremental model update and data management steps specifically include: dividing the steel tower monitoring data at different scales into independent meta-tasks, with each meta-task corresponding to a scale scenario, covering different monitoring distance scenarios, so that meta-learning can learn the model adaptation capability under different scale scenarios.

[0014] The technical solution of the present invention brings at least the following beneficial effects: By dynamically adjusting the receptive field of the attention layer, the accuracy of interference decoupling at different distances is ensured, effectively separating camera shake, wind-induced interference, and the vibration components of the steel tower itself, thus solving the problem of micro-vibration information being submerged. At the same time, through scale normalization and dynamic weight adjustment, effective federated fusion of heterogeneous data from multiple perspectives is achieved. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0016] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1 This application provides a method for identifying the vibration modes and displacements of a steel tower using video tracking, comprising the following steps: The scale calibration step calculates the scale calibration coefficient based on the calibration parameters of the monitoring station and camera, providing a unified quantitative benchmark for end-to-end calibration; The input layer scale calibration step involves resampling the original video frames according to the scale calibration coefficients to unify the spatial scale of the input data and provide standardized input for subsequent processing. The topology optimization layout step for marker points involves extracting topological location features, visual visibility features, and vibration monitoring contribution features of candidate marker points based on a calibrated unified input scale. The features are then weighted and summed to obtain a priority score. Candidate points are sorted from high to low according to the priority score, and the point layout density is adaptively adjusted to provide an effective marking basis for feature extraction. The spatiotemporal frequency domain interference decoupling step dynamically adjusts the receptive field parameters of the attention layer according to the scale calibration coefficient, performs interference decoupling processing on the collected raw monitoring data, removes camera shake and wind-induced environmental interference components, and outputs purified steel tower motion time series data, providing a purified data source for subsequent feature fusion. The multi-view heterogeneous feature federation step involves each edge monitoring node normalizing the scale of its local features based on the local scale calibration coefficient, uploading the normalized local convolutional neural network model parameters to the central node, calculating the fusion weights based on the scale calibration coefficients of each node, and performing a weighted average of the model parameters of each node to obtain the global model parameters, thus completing federated learning and outputting a globally unified motion feature set. The vibration state inference step for low-texture regions involves using the fused feature set and the normalized stiffness constraints of the steel tower members to complete the missing motion features of the low-texture regions based on a knowledge-enhanced graph neural network. This outputs a standardized vibration state feature vector, providing complete state features for subsequent modal identification. The micro-vibration signal modal identification step, based on the state feature vector and a lightweight Transformer model with subspace projection and quantization compression, completes the high-precision identification of the steel tower vibration modal parameters and the entire tower displacement field, and outputs the real-time identification results at the edge end, providing a decision basis for subsequent closed-loop control; The closed-loop monitoring and control steps adaptively adjust the monitoring sampling parameters and the monitoring weights of marked points based on the modality recognition results, optimize the data acquisition quality and monitoring resource allocation, and trigger the model update command based on meta-learning. The incremental model update and data management steps involve iteratively updating model parameters based on a meta-learning continuous incremental learning algorithm, according to model update instructions, while simultaneously storing monitoring data in a hierarchical and categorized manner. This achieves long-term dynamic adaptation of the monitoring model and efficient management of data throughout its entire lifecycle.

[0018] It should also be noted that the reference scale is a fixed and unified scale preset by this invention, specifically 1 pixel corresponds to 0.5 mm of actual spatial scale. This scale is the standard reference scale for monitoring micro-vibration of steel towers, which can effectively capture micro-vibration displacement of steel towers at the level of 0.1 mm. All scale calibration, resampling, receptive field adjustment and feature normalization are based on this as a unified goal. The actual unit space refers to a square actual space area of ​​10cm×10cm. When setting up the marker points, ensure that one marker point is set up in this unit area to achieve uniformity of the actual density of marker points at different distances.

[0019] The reference receptive field is 32×32 pixels, corresponding to an actual space of 16mm×16mm at the reference scale. The formula for adjusting the receptive field at different distances is: current receptive field pixel size = reference receptive field pixel size / scale calibration coefficient, so as to ensure that the actual spatial scale corresponding to the attention layer is consistent at different distances.

[0020] This method solves the scale mismatch problem caused by the optical lever effect. The auxiliary process is as follows: A standard checkerboard calibration plate was deployed on the surface of the steel tower. Multiple sets of calibration plate images were acquired. A camera extrinsic parameter calibration process was executed to extract corner points and calculate pose, obtaining the actual spatial distance from the monitoring station to the steel tower surface. Simultaneously, a camera intrinsic parameter calibration process was executed to solve for the intrinsic parameter matrix and distortion coefficients, obtaining the actual focal length parameters of the camera lens, providing basic physical parameters for coefficient calculation. Input data consisted of the acquired raw video frames; candidate points were the locations of the markers to be deployed on the steel tower surface; motion prior data consisted of synchronously acquired camera motion data; each node represented an edge monitoring camera node; the constraint matrix was the stiffness matrix of the steel tower members; the time series data consisted of purified marker point motion time series data; the sampling parameters were the camera's sampling frame rate and exposure time; and the monitoring data consisted of the steel tower's monitoring data.

[0021] As an optional embodiment, the scale calibration step specifically includes: First, the camera's extrinsic parameter calibration process is executed. A standard checkerboard calibration plate is set up on the surface of the steel tower, and the collected calibration plate image is subjected to sub-pixel level corner detection. Then, based on epipolar geometric constraints, pose calculation is performed to obtain the actual spatial distance from the monitoring station to the surface of the steel tower. Then, the camera's intrinsic parameter calibration process is executed. By fully calibrating the radial and tangential distortion of the camera lens, the error caused by lens distortion is eliminated, and the actual focal length parameter of the camera lens is obtained. Finally, based on the obtained actual distance and focal length parameters, the scale calibration coefficient is calculated.

[0022] The detailed calculation steps for external parameter calibration are as follows: 1. The Harris corner detection algorithm is used to perform a convolution operation on the input calibration board image, calculate the corner response value of each pixel, and extract the initial corner coordinates; 2. Based on the least squares method, the initial corner coordinates are optimized to sub-pixel level. By constructing a local gray-level gradient equation, the high-precision corner position at the sub-pixel level is obtained. 3. Based on the known 3D coordinates of the calibration board and the 2D coordinates in the image, construct the PerspectivenPoint pose problem and construct the reprojection error objective function; 4. The Levenberg-Marquardt iterative optimization algorithm is used to iteratively optimize the objective function, solve for the camera's extrinsic parameter matrix, and then obtain the actual spatial distance from the monitoring station to the steel tower surface.

[0023] The detailed calculation steps for internal parameter calibration are as follows: 1. Collect multiple sets of calibration board images from different angles and distances, and extract the corner coordinates of each set of images; 2. Construct an intrinsic parameter optimization objective function that includes radial distortion polynomial terms and tangential distortion offset terms. The objective function is the sum of the reprojection errors of all corner points. 3. The objective function is optimized using the least squares method to obtain the camera's intrinsic parameter matrix and distortion coefficients, thereby eliminating errors caused by lens distortion and obtaining the actual focal length parameters of the camera lens.

[0024] The final scale calibration coefficient is calculated as the ratio of the actual distance to the focal length. This coefficient represents the actual spatial scale magnification of a single pixel of the camera under long-distance monitoring, quantifies the scale offset caused by the light leverage effect, and provides a unified quantitative benchmark for subsequent end-to-end calibration.

[0025] As an optional embodiment, the input layer scale calibration step specifically includes: Based on the calculated scale calibration coefficients, the original input video frames are resampled using a bilinear interpolation algorithm. The resampling ratio is adaptively adjusted according to the scale calibration coefficients so that the actual spatial scale of the input frames at different monitoring distances is unified to the reference scale.

[0026] The detailed calculation steps for bilinear interpolation are as follows: 1. Calculate the resampling ratio based on the scale calibration coefficient. The resampling ratio is the ratio of the reference scale to the actual scale. 2. For each pixel in the original video frame, calculate its corresponding position in the resampled image according to the resampling ratio, and denot it as (x, y); 3. Obtain the four adjacent pixels around this position, denoted as (x1, y1), (x1, y2), (x2, y1), and (x2, y2), with corresponding gray values ​​of Q11, Q12, Q21, and Q22, respectively; 4. First, perform linear interpolation in the x-direction to obtain...

[0027]

[0028] 5. Then, perform linear interpolation in the y-direction to obtain the grayscale value of the resampled pixel:

[0029] 6. Traverse all pixels to complete resampling, ensuring that edge information of the image is not lost during the resampling process and no additional noise is introduced.

[0030] Ensuring that input data at different distances has a unified spatial scale benchmark allows all subsequent processing steps to use uniform model parameters, eliminating the need to train different models for different distances. This is not a simple image scaling operation, but rather an input layer calibration to address the scale mismatch problem.

[0031] As an optional embodiment, the marker point topology optimization deployment step specifically includes: extracting the topological location features, visual visibility features, and vibration monitoring contribution features of candidate marker points based on the calibrated unified input scale; obtaining a priority score by weighted summation of the features; sorting the candidate points from high to low according to the priority score; for long-distance areas with larger scale calibration coefficients, the deployment density of marker points is higher; by adjusting the density, the number of marker points in a unit of actual space is made uniform; during the deployment process, candidate points are screened to exclude invalid points that are occluded or cannot be observed by the camera, ensuring that all marker points can be effectively observed, thus completing the layered deployment of marker points.

[0032] The detailed calculation steps for priority quantification are as follows: 1. For each candidate point, extract its topological location features, visual visibility features, and vibration monitoring contribution features within the steel tower structure; 2. Perform a weighted summation of the three features to obtain the priority score for each candidate point, with the weights adaptively adjusted according to the importance of the features; 3. Sort the candidate points from highest to lowest priority score and select the candidate point with the highest score as the final marker point. The detailed steps for density adaptive adjustment are as follows: 1. Calculate the marker point layout density in long-distance areas based on the scale calibration coefficient. The larger the scale calibration coefficient, the higher the layout density; 2. Ensure a uniform number of marker points per unit of actual space, so that the marker point density is consistent at different distances, avoiding the problem of excessively sparse marking in long-distance areas and excessively dense marking in short-distance areas.

[0033] This ensures that marker points at different distances can effectively capture the characteristics of micro-vibrations, providing an effective marker basis for subsequent feature extraction, and optimizes the topology to address the scale mismatch problem.

[0034] As an optional embodiment, the spatiotemporal frequency domain interference decoupling step specifically includes: adjusting the receptive field parameters of the attention layer according to the scale calibration coefficient, the size of the receptive field being adjusted according to the scale calibration coefficient so that the actual spatial scale corresponding to the attention receptive field at different distances is unified as the reference scale; taking the synchronously acquired camera motion data as input, the spatial features and frequency domain features of the motion sequence of the video frame are weighted and processed through the adjusted attention layer; in the spatial domain, the interference area caused by camera shake is located; in the frequency domain, the low-frequency wind-induced interference and the high-frequency vibration component of the steel tower itself are distinguished, and the camera shake component, the wind-induced interference component, and the vibration response component of the steel tower itself are separated to complete the decoupling of the interference signal and obtain the steel tower motion time sequence data.

[0035] The detailed calculation steps of the attention layer are as follows: 1. Calculate the receptive field size of the attention layer based on the scale calibration coefficient. The receptive field size is the ratio of the baseline receptive field to the scale calibration coefficient, ensuring that the actual spatial scale corresponding to the attention receptive field at different distances is consistent; 2. Calculation of the spatiotemporal attention layer: For each feature in the spatial dimension, calculate the attention weight of the feature based on its corresponding position and the temporal features of camera motion. The weight is calculated based on the similarity between the feature and the motion prior. The higher the similarity, the lower the weight, thereby suppressing the features of the interference area caused by camera shake; 3. Calculation of the frequency domain attention layer: Perform Fourier transform on the temporal data to obtain the frequency domain components. For each frequency domain component, calculate the attention weight of the component based on its frequency and the correlation with the steel tower vibration. The higher the correlation, the higher the weight, thereby enhancing the high-frequency vibration components of the steel tower itself and suppressing the low-frequency interference components caused by wind; 4. Separate the three components through weighted processing to complete the decoupling of the interference signal. The receptive field of the attention layer is dynamically adjusted to address the scale mismatch problem, solving the problem of receptive field mismatch at long distances and eliminating the influence of interference signals on subsequent processing.

[0036] As an optional embodiment, the multi-view heterogeneous feature federated fusion step specifically includes: each edge camera node performs scale normalization processing on the locally extracted motion features based on the local scale calibration coefficient, unifying the scale of the local features to a reference scale, eliminating scale differences between different nodes, and ensuring that the features of different nodes have a unified scale reference; the normalized local convolutional neural network model parameters are uploaded to the central node; the central node calculates the fusion weight of each node based on the scale calibration coefficient of each camera, with longer-distance nodes having higher fusion weights for larger scale calibration coefficients; and the model parameters of each node are weighted and averaged to obtain the global model parameters.

[0037] The detailed calculation steps of federated fusion are as follows: 1. Local normalization: Each edge camera node scales each dimension of the locally extracted motion features according to the scale calibration coefficient, transforming the feature scale from the local scale to the reference scale to eliminate scale differences between different nodes; 2. Parameter upload: Nodes upload the normalized local convolutional neural network model parameters to the central node without transmitting the original data to protect data privacy; 3. Weight calculation: The central node calculates the fusion weight of each node according to the scale calibration coefficient of each camera. The larger the scale calibration coefficient, the longer the monitoring distance of the node and the greater the noise in the data. Therefore, a higher fusion weight is assigned to enhance the contribution of long-distance data; 4. Weighted fusion: The model parameters of each node are weighted and averaged according to the fusion weight to obtain the global model parameters, completing federated learning.

[0038] As an optional embodiment, the vibration state inference step for low-texture areas specifically includes: normalizing the stiffness matrix of the steel tower members according to the scale calibration coefficient, unifying the scale of the stiffness constraints to the reference scale, and eliminating scale differences in the stiffness constraints; integrating the normalized stiffness constraints into the message passing process of the graph neural network to construct a knowledge-enhanced graph neural network with stiffness constraints, and integrating the prior knowledge of the structure into the model's inference process; in the message passing process of the graph neural network, completing the missing motion features of the low-texture areas according to the stiffness constraints, and completing the motion features of these low-texture areas based on the motion data of adjacent nodes using the prior knowledge of the stiffness constraints; and outputting the vibration state feature vector.

[0039] The detailed computation steps of the graph neural network are as follows: 1. Stiffness normalization: The stiffness matrix of the steel tower members is normalized according to the scale calibration coefficient to unify the scale of stiffness constraints to the reference scale and eliminate scale differences in stiffness constraints; 2. Graph construction: A graph neural network is constructed, where each node corresponds to a marked point on the steel tower, each edge corresponds to a member of the steel tower, and the edge weight corresponds to the normalized member stiffness; 3. Message passing: During message passing, each node collects the features of its neighboring nodes, and then weights the features of neighboring nodes according to the edge weights to update its own features; 4. Feature completion: For nodes in low-texture areas, their own motion features are missing. Therefore, the features of the node are completed using the features of neighboring nodes and the stiffness weights of the edges. According to the stiffness constraints, there is a linear transmission relationship between the motions of neighboring nodes, so the missing features can be completed using the motions of neighboring nodes; 5. Output a standardized vibration state feature vector.

[0040] To address the scale mismatch issue, the stiffness constraints were normalized and integrated into the message passing process, thus resolving the problem of missing features in low-texture regions and ensuring that the motion characteristics of the entire steel tower are complete.

[0041] As an optional embodiment, the micro-vibration signal modal identification step specifically includes: constructing a time series matrix and performing singular value decomposition on the motion time series data of the marked points using a random subspace identification algorithm, completing subspace projection, and extracting modal feature embeddings to map the original high-dimensional time series data to a low-dimensional modal feature space to eliminate the influence of noise; inputting the feature embeddings into a quantized Transformer model, using divergence to calibrate the quantization error during the quantization process to ensure that the accuracy loss of the quantized model at different distances is controlled within an acceptable range; and using this model to complete the identification of the modal parameters of the steel tower and the displacement field of the entire tower.

[0042] The detailed computational steps for modal recognition are as follows: 1. Temporal matrix construction: For the purified marker point motion temporal data, construct the Hankel temporal matrix using a sliding window method, converting the one-dimensional temporal data into a two-dimensional matrix; 2. Singular value decomposition: Perform singular value decomposition on the Hankel matrix, decomposing the matrix into a left singular matrix, a singular value matrix, and a right singular matrix; 3. Subspace projection: Based on the magnitude of the singular values, select the modality-related components, complete the subspace projection, extract the modal feature embedding, and map the original high-dimensional temporal data to a low-dimensional modal feature space to eliminate the influence of noise; 4. Quantization calibration: Perform INT8 quantization on the Transformer model. During the quantization process, use KL divergence to measure the difference in the distribution of the model output before and after quantization. Adjust the quantization parameters according to the difference to ensure that the accuracy loss of the model is controlled within an acceptable range; 5. Inference: Input the feature embedding into the quantized Transformer model to complete the identification of the modal parameters of the steel tower and the displacement field of the entire tower.

[0043] The quantization process was calibrated to address the scale mismatch issue, resolving the inconsistency in quantization accuracy at different distances and achieving high-precision real-time recognition at the edge.

[0044] As an optional embodiment, the closed-loop monitoring and control steps specifically include: acquiring the highest frequency component of the identified vibration signal, adjusting the sampling frame rate to ensure that the sampling frame rate is always more than twice that of the highest frequency component, simultaneously adjusting the exposure time according to the adjustment of the sampling frame rate to ensure stable image brightness, and correcting the monitoring weight of the marker points in the corresponding area according to the identified vibration amplitude distribution. For areas with higher amplitude, more monitoring resources are allocated to improve the monitoring accuracy of the area, while for areas with lower amplitude, the monitoring weight is reduced to save resources, thereby realizing the allocation of monitoring resources. The monitoring parameters are adjusted according to the real-time signal characteristics.

[0045] The detailed calculation steps of closed-loop control are as follows: 1. Frequency adaptive adjustment: Calculate the target sampling frame rate based on the highest frequency component of the identified vibration signal. The target sampling frame rate is twice that of the highest frequency component, satisfying the Nyquist sampling theorem to ensure that high-frequency signals do not alias. 2. Exposure synchronization adjustment: Adjust the camera's exposure time synchronously according to the adjustment of the sampling frame rate. The exposure time is inversely proportional to the sampling frame rate to ensure that the image integration time remains unchanged, thereby ensuring that the image brightness is stable and will not change due to the adjustment of the frame rate. 3. Weight adaptive adjustment: Calculate the monitoring weight of each region based on the identified vibration amplitude distribution. The higher the amplitude, the more intense the vibration in that region, requiring higher monitoring accuracy. Therefore, a higher monitoring weight is assigned, and more monitoring resources are allocated. The lower the amplitude, the lower the weight is assigned to save resources.

[0046] By adaptively adjusting according to the actual characteristics of the signal, the monitoring process is fully adaptively optimized.

[0047] As an optional embodiment, the incremental model update and data management steps specifically include: dividing the steel tower monitoring data at different scales into independent meta-tasks, with each meta-task corresponding to a scale scenario, covering different monitoring distance scenarios, so that meta-learning can learn the model adaptation capability under different scale scenarios.

[0048] During the training process of meta-learning, a scale-adaptive regularization term is added. This regularization term constrains the drift of model parameters and avoids the drift of model parameters due to differences in different scale scenes. When new scale scene monitoring data is input, the model can complete the parameter adaptation without retraining the entire model. Only new data is needed to complete the adaptation, realize model iteration, and ensure that the model can adapt to different scenarios. Meanwhile, the monitoring data of the steel tower is processed in layers according to the scale calibration coefficient. For long-distance monitoring data with a large scale calibration coefficient, a lossless compression algorithm is used for storage to ensure the accuracy of micro-vibration data. For short-distance monitoring data with a small scale calibration coefficient, an adaptive lossy compression algorithm is used for storage to reduce storage overhead. A distributed storage architecture is adopted to classify and store the original monitoring data, motion feature data, model parameters and identification results, so as to realize the management and retrieval of large-scale monitoring data and ensure the traceability and availability of data.

[0049] The detailed calculation steps for incremental updates are as follows: 1. Meta-task partitioning: The steel tower monitoring data at different scales are divided into independent meta-tasks. Each meta-task corresponds to a scale scene, covering different monitoring distances; 2. Regularization term calculation: During the training process of meta-learning, the corresponding model parameters are trained for each meta-task. Then, the L2 norm difference between the model parameters of the current task and the average model parameters of all meta-tasks is calculated. This difference is used as a regularization term for scale adaptation and added to the loss function; 3. Model training: Through this regularization term, the changes in model parameters are constrained to avoid excessive drift of model parameters due to differences in different scale scenes. When monitoring data of new scale scenes are input, the model can quickly adapt the parameters without retraining the entire model.

[0050] The detailed steps of data management are as follows: 1. Hierarchical processing: Based on the scale calibration coefficient, distinguish between long-distance and short-distance monitoring data; 2. Compressed storage: For long-distance data, use the LZ4 lossless compression algorithm to store it to ensure the accuracy of micro-vibration data; for short-distance data, use the adaptive wavelet lossy compression algorithm to store it to reduce storage overhead; 3. Distributed storage: Adopt a distributed storage architecture to classify and store various types of data to achieve efficient management of large-scale data.

[0051] To address the scale mismatch problem, a scale-adaptive regularization term was designed, which solved the problem of model parameter drift during incremental updates, enabling long-term dynamic iteration of the model and efficient management of large-scale monitoring data.

[0052] Additionally, it should be noted that in the above content: 1. Graph structure construction: Node: Each node corresponds to a marked point on the surface of the steel tower. The initial feature of the node is the motion time-series feature vector of the point in the x / y direction, with a dimension of 2×T (T is the time-series length of the sliding window, which is taken as T=30 in this invention).

[0053] Edge: Each edge corresponds to a member of the steel tower. The initial weight of the edge is the normalized member stiffness, calculated by the formula: we=ke / kmax, where ke is the actual stiffness value of the member and kmax is the maximum stiffness value of all members of the steel tower. The normalized weight ranges from 0 to 1.

[0054] 2. Message passing logic: The feature update formula for a node is:

[0055] in Let i be the original feature of node i. Let i be the set of neighboring nodes of node i. The normalized stiffness weights of the edges are used to incorporate the structural priors of the stiffness of the members into the message passing process, ensuring that the motion transmission of adjacent nodes conforms to the laws of structural mechanics.

[0056] 3. Loss function: The training loss function is:

[0057] in MSE loss for motion feature prediction This is a stiffness constraint regularization term used to constrain the matching relationship between the motion difference and stiffness of adjacent nodes. The regularization coefficient is... Take 0.1.

[0058] 3. Supplementary details on the implementation of the lightweight Transformer model with quantization compression 1. Model Structure: Input layer: The input is a 64-dimensional modal feature embedding obtained by subspace projection; Encoder: Employs a 2-layer lightweight Transformer with 2 attention heads. The feedforward network uses depthwise separable convolutions to replace the traditional fully connected layers, compressing the model parameters from the original 12M to 1.2M, making it suitable for edge deployment. Output layer: It is divided into two branches, which output modal parameters (natural frequency, damping ratio, a total of 4 dimensions) and the displacement field of the whole tower (x / y displacement of all marked points, a total of 2×N dimensions, where N is the number of marked points).

[0059] Quantization calibration process: 1. First, perform INT8 linear quantization on the model's weights and activation values. The initial scaling factor is obtained based on the distribution statistics of the weights. 2. Use KL divergence to measure the difference in the distribution of the model output before and after quantization, and iteratively adjust the quantization scaling factor until the KL divergence is less than 0.01.

[0060] To verify the beneficial effects of the present invention, two embodiments of the present invention and one comparative example of a conventional method were set up for comparative experiments: Example 1: Monitoring of a 100-meter-long power transmission tower Monitoring scenario: A 220kV transmission steel tower, 60 meters high, with the monitoring station 100 meters away from the tower. An industrial camera with a resolution of 1920×1080 is used, with an initial sampling frame rate of 30fps.

[0061] Execution process: According to the method of the present invention, the camera calibration is completed to obtain a scale calibration coefficient of 104. The original video frames are resampled and unified to the reference scale. The density of marker points is adaptively adjusted to complete the decoupling of spatiotemporal frequency domain interference. Finally, the modal and displacement recognition is completed through the model of the present invention, and closed-loop control and incremental update are started at the same time.

[0062] Example 2: Monitoring of a communication steel tower at a distance of 200 meters Monitoring scenario: A 120-meter-high 5G communication steel tower, with the monitoring station 200 meters away from the tower. The same 1920×1080 industrial camera is used, with an initial sampling frame rate of 30fps, and two monitoring nodes with different perspectives are deployed.

[0063] Execution process: According to the method of this invention, the scale calibration of the two nodes is completed, the multi-view feature fusion is completed through the federated distillation architecture, the missing features of the low texture area are supplemented through the knowledge-enhanced graph neural network, and the modality and displacement recognition are completed.

[0064] Comparative Example: Conventional Visual Inspection Methods Monitoring scenario: The communication steel tower with a distance of 200 meters is exactly the same as in Example 2. The existing conventional visual monitoring method is adopted: the optical flow method is directly used to track the marker points. There is no scale calibration, no interference decoupling, no federated fusion, and the original image is directly processed to complete the modality recognition. The core performance metrics of the three schemes are compared below:

[0065] 2. Error comparison at different distances For different monitoring distances, the displacement measurement errors of the method of this invention and conventional methods were compared, and the results are as follows:

[0066] Experimental conclusions The experimental data show that: The method of this invention has a displacement measurement error that is much lower than that of conventional methods under long-distance monitoring.

[0067] Working principle First, through the camera's extrinsic and intrinsic parameter calibration processes, the actual spatial distance from the monitoring station to the steel tower and the actual focal length parameters of the camera lens are obtained. Based on these parameters, the scale calibration coefficient is calculated. This coefficient is used to quantify the magnification factor of the actual spatial scale corresponding to the camera pixel under long-distance monitoring, which is the scale offset caused by the optical lever effect, providing a unified quantitative benchmark for subsequent end-to-end calibration.

[0068] Based on the scale calibration coefficient, the original input video frames are resampled pixel by pixel to unify the input data at different monitoring distances to the reference scale and ensure the consistency of the model input. At the same time, based on the calibrated input scale, the candidate marker points on the steel tower surface are prioritized and quantified. The marker density is adaptively increased for long-distance areas to ensure that the number of marker points in a unit of actual space is uniform, thus solving the problem of sparse markers in long-distance areas and the inability to effectively capture micro-vibrations.

[0069] The receptive field parameters of the attention layer are dynamically adjusted based on the scale calibration coefficient to ensure that the actual spatial scale corresponding to the attention receptive field is consistent at different distances. Then, through the spatiotemporal frequency domain attention layer, the camera shake, wind-induced interference and the vibration components of the steel tower itself are accurately separated to complete the interference decoupling and output purified motion time series data. Each edge monitoring node performs scale normalization on the locally extracted motion features based on the local scale calibration coefficient. The central node dynamically adjusts the fusion weight according to the scale coefficient of each node to complete federated learning and achieve effective integration of heterogeneous data.

[0070] The stiffness matrix of the steel tower members is normalized according to the scale calibration coefficient and integrated into the message passing process of the graph neural network. The missing motion features in the low texture area are completed by stiffness constraints, and a complete vibration state feature vector is output. Then, the subspace projection is completed by random subspace recognition algorithm, modal features are extracted and embedded, and input into the quantized and calibrated Transformer model to complete the identification of the steel tower vibration modal parameters and the displacement field of the whole tower, realizing real-time high-precision identification at the edge end.

[0071] Based on the modality recognition results, the monitoring sampling frame rate and the monitoring weight of the marked points are adaptively adjusted to dynamically optimize the allocation of monitoring resources. At the same time, monitoring data at different scales are divided into independent meta-tasks, and scale-adaptive regularization terms are added to constrain model parameter drift, so as to realize long-term dynamic iteration of the model. A hierarchical storage strategy is adopted for monitoring data at different scales to reduce storage overhead while ensuring data accuracy, so as to achieve efficient management of monitoring data throughout the entire life cycle.

[0072] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for identifying vibration modes and displacements of a steel tower using video tracking, characterized in that, Includes the following steps: scale The calibration process calculates the scale calibration coefficient based on the calibration parameters of the monitoring station and camera, providing a unified quantitative benchmark for end-to-end calibration. The input layer scale calibration step involves resampling the original video frames according to the scale calibration coefficients to unify the spatial scale of the input data and provide standardized input for subsequent processing. The topology optimization layout step for marker points involves extracting topological location features, visual visibility features, and vibration monitoring contribution features of candidate marker points based on a calibrated unified input scale. The features are then weighted and summed to obtain a priority score. Candidate points are sorted from high to low according to the priority score, and the point layout density is adaptively adjusted to provide an effective marking basis for feature extraction. The spatiotemporal frequency domain interference decoupling step dynamically adjusts the receptive field parameters of the attention layer according to the scale calibration coefficient, performs interference decoupling processing on the collected raw monitoring data, removes camera shake and wind-induced environmental interference components, and outputs purified steel tower motion time series data, providing a purified data source for subsequent feature fusion. The multi-view heterogeneous feature federation step involves each edge monitoring node performing scale normalization of local features based on local scale calibration coefficients, uploading the normalized local convolutional neural network model parameters to the central node, calculating the fusion weights based on the scale calibration coefficients of each node, and performing a weighted average of the model parameters of each node to obtain the global model parameters, completing federated learning, and outputting a globally unified set of motion features to provide a fusion feature foundation for subsequent state inference. The vibration state inference step for low-texture regions involves using the fused feature set and the normalized stiffness constraints of the steel tower members to complete the missing motion features of the low-texture regions based on a knowledge-enhanced graph neural network. This outputs a standardized vibration state feature vector, providing complete state features for subsequent modal identification. The micro-vibration signal modal identification step, based on the state feature vector and a lightweight Transformer model with subspace projection and quantization compression, completes the high-precision identification of the steel tower vibration modal parameters and the entire tower displacement field, and outputs the real-time identification results at the edge end.

2. The method for identifying the vibration modes and displacement of a steel tower using video tracking according to claim 1, characterized in that, The scale calibration step specifically includes: The camera's extrinsic parameter calibration process is executed by setting up a standard checkerboard calibration plate on the steel tower surface, performing sub-pixel-level corner detection on the acquired calibration plate image, and then performing pose calculation based on epipolar geometric constraints to obtain the actual spatial distance from the monitoring station to the steel tower surface. The camera's intrinsic parameter calibration process is executed. By fully calibrating the radial and tangential distortion of the camera lens, errors caused by lens distortion are eliminated, and the actual focal length parameters of the camera lens are obtained. Based on the obtained actual distance and focal length parameters, the scale calibration coefficient is calculated.

3. The method for identifying the vibration modes and displacement of a steel tower using video tracking according to claim 1, characterized in that, The input layer scale calibration step specifically includes: Based on the calculated scale calibration coefficients, the original input video frames are resampled using a bilinear interpolation algorithm. The resampling ratio is adaptively adjusted according to the scale calibration coefficients so that the actual spatial scale of the input frames at different monitoring distances is unified to the reference scale.

4. The method for identifying the vibration modes and displacement of a steel tower using video tracking according to claim 1, characterized in that, The specific steps of the marker point topology optimization deployment include: based on the calibrated unified input scale, extracting the topological location features, visual visibility features, and vibration monitoring contribution features of candidate marker points; weighted summing of the features to obtain a priority score; sorting the candidate points from high to low according to the priority score; for long-distance areas with larger scale calibration coefficients, the deployment density of marker points is higher; by adjusting the density, the number of marker points in a unit of actual space is made uniform; during the deployment process, candidate points are screened to exclude invalid points that are obscured or cannot be observed by the camera, ensuring that all marker points can be effectively observed, thus completing the layered deployment of marker points.

5. The method for identifying the vibration modes and displacement of a steel tower using video tracking according to claim 1, characterized in that, The specific steps of the spatiotemporal frequency domain interference decoupling include: adjusting the receptive field parameters of the attention layer according to the scale calibration coefficient, and adjusting the size of the receptive field according to the scale calibration coefficient so that the actual spatial scale corresponding to the attention receptive field at different distances is unified as the reference scale; taking the synchronously acquired camera motion data as input, and through the adjusted attention layer, weighting the spatial features of the video frame and the frequency domain features of the motion sequence, in the spatial domain, locating the interference area caused by camera shake, and in the frequency domain, distinguishing the low-frequency wind-induced interference and the high-frequency vibration component of the steel tower itself, separating the camera shake component, the wind-induced interference component and the vibration response component of the steel tower itself, completing the decoupling of the interference signal, and obtaining the steel tower motion sequence data.

6. The method for identifying the vibration modes and displacement of a steel tower using video tracking according to claim 1, characterized in that, The multi-view heterogeneous feature federated fusion step specifically includes: each edge camera node performs scale normalization processing on the locally extracted motion features based on the local scale calibration coefficient, unifying the scale of the local features to a reference scale, eliminating scale differences between different nodes, and ensuring that the features of different nodes have a unified scale reference; the normalized local convolutional neural network model parameters are uploaded to the central node; the central node calculates the fusion weight of each node based on the scale calibration coefficient of each camera, with longer-distance nodes having higher fusion weights for larger scale calibration coefficients; and the model parameters of each node are weighted and averaged to obtain the global model parameters.

7. The method for identifying the vibration modes and displacement of a steel tower using video tracking according to claim 1, characterized in that, The vibration state inference steps for low-texture regions specifically include: normalizing the stiffness matrix of the steel tower members according to the scale calibration coefficient, unifying the scale of the stiffness constraints to the reference scale, and eliminating scale differences in the stiffness constraints; integrating the normalized stiffness constraints into the message passing process of the graph neural network to construct a knowledge-enhanced graph neural network with stiffness constraints, and integrating the prior knowledge of the structure into the model's inference process; in the message passing process of the graph neural network, completing the missing motion features of the low-texture regions according to the stiffness constraints, and completing the motion features of these low-texture regions based on the motion data of adjacent nodes using the prior knowledge of the stiffness constraints; and outputting the vibration state feature vector.

8. The method for identifying the vibration modes and displacement of a steel tower using video tracking according to claim 1, characterized in that, The specific steps of the micro-vibration signal modal identification include: constructing a time series matrix and performing singular value decomposition on the motion time series data of the marked points using a random subspace identification algorithm, completing subspace projection, and extracting modal feature embeddings. This maps the original high-dimensional time series data to a low-dimensional modal feature space, eliminating the influence of noise. The embedded features are then input into a quantized Transformer model. The quantization process uses divergence to calibrate the quantization error, ensuring that the accuracy loss of the quantized model at different distances is controlled within an acceptable range. Through this model, the modal parameters of the steel tower and the displacement field of the entire tower are identified.

9. The method for identifying the vibration modes and displacement of a steel tower using video tracking according to claim 1, characterized in that, It also includes closed-loop monitoring and control steps, specifically including: acquiring the highest frequency component of the identified vibration signal, adjusting the sampling frame rate to ensure that the sampling frame rate is always more than twice that of the highest frequency component, simultaneously adjusting the exposure time according to the adjustment of the sampling frame rate to ensure stable image brightness, and correcting the monitoring weight of the marker points in the corresponding areas according to the identified vibration amplitude distribution. For areas with higher amplitude, more monitoring resources are allocated to improve the monitoring accuracy of the area, while for areas with lower amplitude, the monitoring weight is reduced to save resources, thus realizing the allocation of monitoring resources. The monitoring parameters are adjusted according to the real-time signal characteristics.

10. The method for identifying the vibration modes and displacement of a steel tower using video tracking according to claim 1, characterized in that, It also includes incremental model updates and data management steps, specifically: dividing steel tower monitoring data at different scales into independent meta-tasks, with each meta-task corresponding to a scale scenario, covering different monitoring distance scenarios, so that meta-learning can learn the model adaptation capability under different scale scenarios.

Images