Vibration monitoring and displacement identification method for towering steel structure based on machine vision

Abstract

The invention discloses a vibration monitoring and displacement identification method for a towering steel structure based on machine vision, and relates to the technical field of towering steel structures. Comprising the following steps: S1, visual acquisition and reference calibration: arranging visual acquisition equipment in the height direction and the circumferential direction of the towering steel structure according to mechanical characteristics of a vertical stiffness mutation region and a dynamic response sensitive part of a geometric asymmetric distribution characteristic of the structure; a three-dimensional coordinate system is established based on a structure installation reference surface, fixed reference monitoring points are arranged in a tower foot symmetric area and a mast antenna root, three-dimensional coordinate data of the reference points are synchronously collected, the collection range completely covers a node concentration area and a key stress part of a structure transparent area, and a continuous image sequence of the structure surface is captured in real time; the resolution and the sampling frequency of the visual acquisition equipment are determined according to the inherent frequency of the structure and the vibration response characteristics, and calibration is completed through reference point coordinates after the equipment is installed; s2, carrying out image preprocessing and feature enhancement; S3, carrying out feature point extraction and trajectory tracking; S4, carrying out vibration displacement calculation and twin fusion; and S5, carrying out multi-source verification and structural state linkage treatment.

Description

Technical Field

[0001] This invention relates to the field of tall steel structure technology, and in particular to a method for vibration monitoring and displacement identification of tall steel structures based on machine vision. Background Technology

[0002] Modern tall steel structures (such as television towers and communication towers) have become iconic urban buildings. Their designs often adopt a personalized "one tower, one design" approach, frequently featuring complex structural forms such as lattice-shaped cylindrical bodies, multi-segment open areas, and top mast antennas. These structures generally exhibit characteristics such as geometrically asymmetrical vertical stiffness abrupt changes, dynamic coupling between the mast and the main body (whiplash effect), which imposes stringent requirements on safety monitoring during construction and operation and maintenance.

[0003] There are several technical challenges in the current field of monitoring tall steel structures: First, the complexity of the structure makes monitoring difficult, and conventional linear analysis struggles to accurately capture the dynamic response caused by geometric asymmetry and abrupt changes in stiffness. Furthermore, there is a lack of sophisticated monitoring methods for the whiplash effect of mast antennas. Second, existing monitoring models are mostly based on idealized assumptions, which deviate from actual construction factors such as node construction errors and welding residual stress, leading to discrepancies between monitoring results and the actual structural state. Third, the ability to monitor multiple disasters in a coupled manner is insufficient. Existing research often focuses on single seismic effects, failing to fully consider the synergistic effects of wind load, temperature deformation, and earthquakes, making it unsuitable for complex service environments. Fourth, traditional monitoring methods rely on manual inspections and single sensor data, resulting in issues such as monitoring blind spots, data lag, and weak anti-interference capabilities. They also lack dynamic linkage between the physical structure and the virtual model, making it difficult to achieve full-cycle intelligent management and control. Fifth, machine vision monitoring technology faces bottlenecks in application. Single-resolution optical flow algorithms are prone to losing tracking points due to rapid structural vibrations and changes in surface features, and environmental interference such as dust and lighting changes further affect monitoring accuracy. Summary of the Invention

[0004] This invention provides a method for vibration monitoring and displacement identification of tall steel structures based on machine vision. The technical solution is as follows: A machine vision-based method for vibration monitoring and displacement identification of tall steel structures includes the following steps: S1 Visual Acquisition and Benchmark Calibration: Visual acquisition equipment is deployed along the height and circumference of the tall steel structure, based on the mechanical properties of the vertical stiffness abrupt change areas and dynamic response sensitive parts, according to the structural geometric asymmetry distribution characteristics. A three-dimensional coordinate system is established based on the structural installation reference surface, and fixed benchmark monitoring points are set up in the symmetrical area of ​​the tower foot and at the base of the mast antenna. The three-dimensional coordinate data of the benchmark points are collected synchronously. The acquisition range completely covers the structural open area, node concentration area and key stress parts. Continuous image sequences of the structural surface are captured in real time. The resolution and sampling frequency of the visual acquisition equipment are determined based on the natural frequency and vibration response characteristics of the structure. After the equipment is installed, calibration is completed through the benchmark coordinates. S2 Image Preprocessing and Feature Enhancement: Noise suppression, illumination equalization, and spatial registration operations are sequentially performed on the acquired image sequences; signal interference caused by equipment vibration and environmental dust is eliminated through an adaptive filtering algorithm; grayscale stretching technology is used to correct brightness deviations caused by illumination changes; precise alignment of continuous frame images is achieved based on the three-dimensional coordinates of the reference monitoring point; grayscale gradient and geometric recognition of mechanically stable parts such as weld seam edge nodes and plate corners are enhanced; the filter kernel parameters of the adaptive filtering algorithm are dynamically adjusted according to the intensity of environmental interference; and the grayscale range of grayscale stretching is determined based on the reflective properties of the structural surface material. S3 Feature Point Extraction and Trajectory Tracking: In the preprocessed image, points with significant geometric features and stable material properties on the structural surface are selected. These points meet the requirements of structural mechanical stability and cover weld edge node plate corners and main load-bearing member connections. An optimized optical flow tracing algorithm is used to track the motion trajectory of feature points in consecutive image frames, and abnormal trajectory points caused by non-structural disturbances are automatically removed. The determination of abnormal trajectory points is based on the abrupt change threshold of the feature point's motion velocity and acceleration, and the threshold is determined by the structural dynamic response characteristics. S4 Vibration Displacement Calculation and Twin Fusion: Based on the motion trajectory of feature points, combined with the geometric dimensions, material mechanical properties, and calibration data of the tall steel structure and visual acquisition equipment, a mapping relationship between pixel coordinates and physical coordinates is established; the vibration parameters and displacement data of each monitoring point of the structure are calculated through a deep learning model, and the calculation results are transmitted to the digital twin model of the tall steel structure in real time to realize the real-time synchronous mapping between the physical structure state and the virtual model; S5 Multi-Source Verification and Structural Status Linkage: By deploying strain gauges and accelerometers, structural stress data and vibration acceleration data are collected and cross-verified with machine vision calculation results. Based on the verified monitoring data, combined with the structural modal analysis results, material mechanical properties, and seismic response laws, the corresponding handling process is automatically initiated, providing data support for construction process adjustments and disaster emergency response. The cross-verification is performed through a data correlation analysis algorithm, and the correlation threshold is determined by the structural monitoring accuracy requirements.

[0005] Beneficial effects Adapting to the complex characteristics of tall steel structures and solving core monitoring challenges: By optimizing the optical flow tracing algorithm and multi-scale pyramid strategy, the problem of trajectory tracking loss caused by abrupt changes in geometric asymmetric stiffness is specifically addressed, ensuring the continuity of feature point trajectory monitoring and overcoming the limitations of traditional visual monitoring in complex structures; Combining the dynamic coupling characteristics of the mast antenna and the main structure, the monitoring deviation caused by the whiplash effect is corrected through a digital twin model, improving the monitoring accuracy of special structural parts.

[0006] Enhanced multi-condition adaptability to cover complex service environments: Through the seismic condition adaptation module, the calculation logic is adjusted for different disaster scenarios such as far-field long-period near-field pulse earthquakes. Combined with seismic vulnerability curve verification data, it solves the problem of insufficient multi-disaster coupled monitoring in existing technologies. The scene-adaptive module for image preprocessing is specifically designed to deal with environmental interference such as dust, light changes, rain, and snow, ensuring monitoring stability in complex environments such as strong northwest winds.

[0007] Enhancing the reliability and integrity of monitoring data: The multi-source verification mechanism reduces the systematic error of a single monitoring method through cross-verification and error compensation of machine vision strain gauge and accelerometer data, ensuring the authenticity and reliability of the data; Automatic parameter control throughout the entire process dynamically adjusts the equipment and algorithm parameters based on the accuracy of vibration intensity and environmental interference calculation, ensuring the stable operation of the monitoring system under changing structural dynamic response scenarios and avoiding data gaps.

[0008] Supporting full-cycle safety management and adapting to actual engineering needs: The structural condition linkage response process automatically generates a response mechanism based on the structural modal analysis results and material properties, achieving seamless integration of monitoring data and construction adjustment emergency response, solving the problem of lagging traditional monitoring response; the technical solution can be directly applied to the entire construction and operation and maintenance phase, providing data support for structural health assessment process optimization and seismic assessment, and has significant industrial application value. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A process flow diagram provided for an embodiment of this application. Detailed Implementation

[0011] The technical solution provided in this application will now be described in conjunction with the accompanying drawings.

[0012] To facilitate understanding of the embodiments of this application, the following points will be explained first: First, in this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates an "or" relationship between the preceding and following related objects, but does not exclude the possibility of indicating an "and" relationship. The specific meaning can be understood in conjunction with the context. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, "at least one of a, b, or c" can represent: a, b, c; a and b; a and c; b and c; or a and b and c. Here, a, b, and c can be single or multiple.

[0013] Second, in this application, the use of prefixes such as "first," "second," etc., is merely for the purpose of distinguishing and describing different things belonging to the same name category, and does not constrain the order, size, or quantity of things. For example, "first message" and "second message" are simply different messages, and there is no temporal, size, or priority relationship between them.

[0014] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0015] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0016] like Figure 1 As shown, the vibration monitoring and displacement identification method for tall steel structures based on machine vision includes the following steps: S1 Visual Acquisition and Benchmark Calibration: Visual acquisition equipment is deployed along the height and circumference of the tall steel structure, based on the mechanical properties of the vertical stiffness abrupt change areas and dynamic response sensitive parts, according to the structural geometric asymmetry distribution characteristics. A three-dimensional coordinate system is established based on the structural installation reference surface, and fixed benchmark monitoring points are set up in the symmetrical area of ​​the tower foot and at the base of the mast antenna. The three-dimensional coordinate data of the benchmark points are collected synchronously. The acquisition range completely covers the structural open area, node concentration area and key stress parts. Continuous image sequences of the structural surface are captured in real time. The resolution and sampling frequency of the visual acquisition equipment are determined based on the natural frequency and vibration response characteristics of the structure. After the equipment is installed, calibration is completed through the benchmark coordinates. S2 Image Preprocessing and Feature Enhancement: Noise suppression, illumination equalization, and spatial registration operations are sequentially performed on the acquired image sequences; signal interference caused by equipment vibration and environmental dust is eliminated through an adaptive filtering algorithm; grayscale stretching technology is used to correct brightness deviations caused by illumination changes; precise alignment of continuous frame images is achieved based on the three-dimensional coordinates of the reference monitoring point; grayscale gradient and geometric recognition of mechanically stable parts such as weld seam edge nodes and plate corners are enhanced; the filter kernel parameters of the adaptive filtering algorithm are dynamically adjusted according to the intensity of environmental interference; and the grayscale range of grayscale stretching is determined based on the reflective properties of the structural surface material. S3 Feature Point Extraction and Trajectory Tracking: In the preprocessed image, points with significant geometric features and stable material properties on the structural surface are selected. These points meet the requirements of structural mechanical stability and cover weld edge node plate corners and main load-bearing member connections. An optimized optical flow tracing algorithm is used to track the motion trajectory of feature points in consecutive image frames, and abnormal trajectory points caused by non-structural disturbances are automatically removed. The determination of abnormal trajectory points is based on the abrupt change threshold of the feature point's motion velocity and acceleration, and the threshold is determined by the structural dynamic response characteristics. S4 Vibration Displacement Calculation and Twin Fusion: Based on the motion trajectory of feature points, combined with the geometric dimensions, material mechanical properties, and calibration data of the tall steel structure and visual acquisition equipment, a mapping relationship between pixel coordinates and physical coordinates is established; the vibration parameters and displacement data of each monitoring point of the structure are calculated through a deep learning model, and the calculation results are transmitted to the digital twin model of the tall steel structure in real time to realize the real-time synchronous mapping between the physical structure state and the virtual model; S5 Multi-Source Verification and Structural Status Linkage: By deploying strain gauges and accelerometers, structural stress data and vibration acceleration data are collected and cross-verified with machine vision calculation results. Based on the verified monitoring data, combined with the structural modal analysis results, material mechanical properties, and seismic response laws, the corresponding handling process is automatically initiated, providing data support for construction process adjustments and disaster emergency response. The cross-verification is performed through a data correlation analysis algorithm, and the correlation threshold is determined by the structural monitoring accuracy requirements.

[0017] As an optional embodiment, the optimized optical flow tracing algorithm in step S3 is executed according to the following steps: First, a Gaussian pyramid is constructed by multi-scale downsampling of the preprocessed image. Then, a Laplacian pyramid is generated by the difference between adjacent scale images. The pyramid levels are determined by the geometric asymmetry of the towering steel structure and the natural frequencies obtained by the multiple Ritz vector method. Combining the periodicity of structural vibration with the modal decomposition results, a consistency constraint is applied to the motion trajectory of feature points. When the trajectory deviates from the periodic trend, local re-tracking is initiated based on the trajectory data of the previous period and the modal parameters of the modal analysis, while the pixel coordinate deviation of the feature points is corrected. This algorithm is closely related to the feature point selection logic in step S3. Through the synergistic effect of multi-scale tracking and periodic constraints, it solves the problem of vibration trajectory deviation caused by abrupt changes in the geometric asymmetric stiffness of tall steel structures, ensuring the continuity and accuracy of trajectory tracking.

[0018] As an optional embodiment, the deep learning model in step S4 is constructed and applied according to the following steps: First, the spatial correlation features of the motion trajectory of feature points are extracted through the multi-head attention mechanism of the Transformer model, focusing on capturing the trajectory differences between the structural node area and the mid-span of the member due to stiffness differences. Then, the temporal dynamic characteristics of the trajectory are captured by the gating unit of the long short-term memory network, which enhances the identification of the periodic pattern of the vibration signal and the abrupt change characteristics under seismic action. During model training, a metaheuristic NRBO optimization algorithm is introduced to automatically adjust the network hyperparameters. The hyperparameters to be adjusted include the number of attention heads, the number of hidden layer neurons, and the learning rate. The hyperparameter adjustment is based on the spatial distribution characteristics of the temporal continuity displacement data of the vibration signal of the tall steel structure and the seismic vulnerability curve data, and automatically matches the monitoring requirements of members with different heights and stress types. The application process of this model is directly related to the optimized optical flow trajectory data in step S3. By fitting the mapping relationship between vibration parameters and displacement through training set data, the calculation deviation under complex stress conditions is corrected.

[0019] As an optional embodiment, the digital twin model in step S4 is constructed and fused in the following manner: the three-dimensional geometric data of the tall steel structure is obtained by point cloud scanning technology, and the digital twin model is reconstructed by combining BIM technology and elastoplastic time history analysis results. The model includes the geometric parameters, material properties, construction stage information, design safety thresholds and seismic response characteristic data of the structure, and also incorporates the dynamic coupling effect data of the mast antenna and the main structure. The fusion of digital twin model and monitoring data adopts a two-way real-time synchronous mapping method. On the one hand, it receives the solution data from step S4, adjusts the synchronization period according to the structural stress changes and seismic response characteristics during the construction stage, updates the vibration state and displacement distribution of the virtual structure in real time, and focuses on correcting the monitoring deviation caused by the whip effect of the mast antenna. On the other hand, by using virtual simulation to deduce the stress trend of the structure, the system outputs suggestions for adjusting the sampling frequency feature point extraction density of the visual acquisition device, which forms a loop with the trajectory tracking process in step S3 and the solution process in step S4, thus achieving two-way guidance between the virtual and real worlds.

[0020] As an optional embodiment, the multi-source verification in step S5 is performed according to the following steps: First, based on the monitoring accuracy characteristics, response speed, and environmental adaptability of machine vision strain gauges and accelerometers, the verification weights of the three types of data are dynamically allocated. The weight allocation is automatically adjusted according to the measurement deviation law of each monitoring method under the coupled action of seismic wind load and temperature deformation. When a certain type of monitoring data deviates due to environmental interference, historical monitoring data stored in the digital twin model and structural elastoplastic time history analysis data are called up for error compensation; Then, the stress data, vibration acceleration data, and displacement data calculated by machine vision are correlated and analyzed. The reliability of the monitoring results is verified by combining the seismic vulnerability curve data. The systematic error of a single monitoring method is reduced by the data fusion algorithm. This verification process is closely linked to the digital twin model data storage function in step S4, adapting to monitoring deviations caused by structural material nonlinearity and construction errors.

[0021] As an optional embodiment, the image preprocessing in step S2 sets up a scene-adaptive processing module, which handles different environmental interferences according to the following steps: When encountering dust interference, morphological operations are first used to remove floating dust noise, and then the pixel information of the feature point area is repaired by the image restoration algorithm. The repair process adjusts the algorithm parameters according to the gray-level gradient distribution law of the feature points and the material properties of the structural surface. When encountering changes in lighting, the system first performs adaptive threshold segmentation to adjust the grayscale range, and then corrects the brightness deviation through grayscale equalization technology to ensure that the grayscale contrast between the feature points and the background meets the recognition requirements. When encountering rain and snow interference, the rain and snow pixels are first identified by analyzing the pixel grayscale difference, and then the noise is removed by the filtering algorithm. The size of the filtering kernel is adjusted according to the pixel size and surface geometric features of the rain and snow particles. The module parameters are adjusted based on real-time monitoring data of the on-site environment to ensure that the feature points in the preprocessed image meet the extraction and tracking requirements of step S3. This module is linked with the working status monitoring data of the visual acquisition device in step S1.

[0022] As an optional embodiment, the structural state linkage processing of S5 is performed according to the following steps: Based on the modal analysis results of the tall steel structure, the material mechanical properties and seismic response laws are automatically generated by the digital twin model to trigger the dynamic response of the structure. The conditions are adjusted in real time according to the stress state and the intensity of multi-hazard coupling effects as the structure is under construction. When the monitoring data reaches this condition, the system automatically initiates the handling process, and synchronizes the relevant technical and management positions through audio-visual prompts and multi-terminal data push. At the same time, it triggers encrypted sampling of visual acquisition equipment in the local area and elastic-plastic time history simulation of digital twin model to verify the authenticity of structural dynamic response. When the monitoring data continues to exceed the condition and the simulation shows that the structure has a damage evolution trend, the system automatically generates suggestions for adjusting the construction process and structural reinforcement schemes. The multi-source verification results of the handling process and step S5 are closely linked with the digital twin model simulation data of step S4, ensuring the pertinence and reliability of the handling, and adapting to the structural safety monitoring needs under the coupled effects of seismic wind load and temperature deformation.

[0023] As an optional embodiment, the deployment of the visual acquisition device in step S1 is performed according to the following steps: Along the height of the tall steel structure, monitoring layers are divided according to the concentration of nodes in the main load-bearing members and the abrupt change in vertical stiffness. The spacing between monitoring layers is determined based on the attenuation law of the structural dynamic response and the results of multiple Ritz vector method modal analysis. Visual acquisition devices are deployed symmetrically around each floor. The installation positions of the devices avoid structural obstruction areas, and the installation angles are adjusted according to the tilt angle of the plane where the feature points are located to ensure that the optical axis of the lens is perpendicular to the plane of the feature points. During the construction phase, the positions and acquisition ranges of the devices are adjusted according to the progress of structural assembly and changes in vibration-sensitive areas, with a focus on strengthening the monitoring coverage of the base of the mast antenna and the open area. This deployment strategy is linked to the precise location of the processing area in step S7, while also adapting to the feature point extraction requirements of step S3, avoiding monitoring blind spots, and adapting to the monitoring requirements of geometrically asymmetric structures.

[0024] As an optional embodiment, an automatic parameter adjustment step for the entire process is also included, which is specifically executed as follows: Status monitoring modules are set up in the data processing unit of the visual acquisition device and the digital twin platform to collect the device's working parameters, algorithm running parameters and model synchronization parameters in real time. The control unit receives environmental interference data from step S6, trajectory tracking effect from step S3, and solution accuracy feedback from step S4, and automatically adjusts the parameters of each step. When the vibration intensity increases, the sampling frequency and feature point extraction density are increased, while the exposure time is adjusted to reduce motion blur, and the sampling frequency is adjusted to match the natural frequency obtained from the structural modal analysis; when the environmental interference is severe, the preprocessing algorithm parameters are optimized to enhance the anti-interference ability. When the calculation error exceeds the allowable range, the hyperparameters of the deep learning model are adjusted to correct the calculation deviation. The hyperparameter adjustment is based on the seismic vulnerability analysis data. This control step covers all previous process steps, realizes the adaptation of the entire process of acquisition, processing, calculation and fusion, and ensures the stability of the monitoring system under the scenarios of structural dynamic response changes and environmental disturbance fluctuations.

[0025] As an optional embodiment, the vibration displacement calculation in step S4 further includes a seismic condition adaptation module, which is executed according to the following steps: For different seismic conditions such as far-field long-period earthquakes and near-field pulse earthquakes, the feature extraction weights of the deep learning model are adjusted in combination with the dynamic response characteristics and modal decomposition results of tall steel structures to strengthen the solution logic of the coupling relationship between long-period vibration signals and displacement. During the calculation process, the structural deformation trend under earthquake action is simulated by a digital twin model, and the calculation results are supplemented and verified by combining the earthquake vulnerability curve. Simultaneously, structural seismic vulnerability analysis data are introduced to correct displacement calculation errors caused by the mast antenna whip effect. This module is closely related to the basic solution logic of step S4 and the multi-source verification mechanism of step S5, which expands the applicability of the method in earthquake disaster monitoring scenarios and adapts to the seismic assessment needs of tall steel structures.

[0026] Example; This method achieves vibration monitoring and displacement identification of tall steel structures through a progressive operation involving visual acquisition, image preprocessing, feature point tracking, displacement calculation, and multi-source verification. Data flows unidirectionally between steps via data interfaces, with the output data of preceding steps serving as input data for subsequent steps. Parameter verification rules ensure data consistency and validity during the data flow process.

[0027] II. Specific Implementation Details of Each Step (I) Implementation of S1 Visual Acquisition and Benchmark Calibration Selection and Deployment of Visual Acquisition Equipment Industrial-grade cameras were selected for the visual acquisition equipment, with resolution and frame rate determined based on structural vibration frequencies and monitoring accuracy requirements. During deployment, equipment was installed symmetrically around the perimeter of each layer according to the monitoring layer divisions. Installation locations were determined through on-site surveying to ensure the acquisition range covered the structural open areas, concentrated node areas, and critical stress points. After installation, calibration was performed using the 3D coordinate data of the reference monitoring points. The calibration process involved calculating the equipment's internal and external parameters using a coordinate system transformation algorithm, and the parameter results were stored in the data processing unit.

[0028] Benchmark monitoring point setup and data acquisition The benchmark monitoring points are fixed locations selected from the symmetrical area at the tower base and the base of the mast antenna. The selection of these points is based on the geometric characteristics of the structure to ensure that the points remain stable throughout the monitoring period. A total station is used to collect the three-dimensional coordinate data of the benchmark points. The data acquisition process is performed according to standard measurement procedures. After error correction, the collected data is used to establish a global three-dimensional coordinate system.

[0029] Image sequence acquisition The visual acquisition device acquires a sequence of images of the structural surface at a set sampling frequency. The image format is a lossless compression format. During the acquisition process, the status monitoring module monitors the working status of the device in real time and triggers an alarm mechanism when an abnormality occurs.

[0030] (II) Implementation of S2 Image Preprocessing and Feature Enhancement Noise suppression operation An adaptive filtering algorithm is used to process the image sequence, and the filter kernel parameters are dynamically adjusted according to the intensity of environmental interference. When there is dust interference, the filter kernel size is increased; when there is equipment vibration interference, the filter kernel shape is adjusted. The filtering algorithm is executed through convolution operations, and the results are output to the illumination equalization step.

[0031] Light equalization operation Gray-scale stretching is used to correct image brightness deviations, and the stretching range is determined based on the reflective properties of the surface material. By calculating the image's gray-scale histogram, upper and lower thresholds for stretching are determined, and gray-scale transformation is performed to ensure that the gray-scale contrast of feature point regions in the image meets recognition requirements.

[0032] Spatial registration operation Based on the 3D coordinate data of the reference monitoring points, the alignment of consecutive frames of images is achieved through an image registration algorithm. The registration algorithm adopts the feature point matching method, using the pixel position of the reference monitoring points in the image as a reference, calculating the transformation matrix between image frames, and achieving image alignment through matrix transformation, with the alignment error controlled within a preset range.

[0033] (III) Implementation of S3 Feature Point Extraction and Trajectory Tracking Feature point filtering A corner detection algorithm is used to screen feature points on the structural surface. The screening rules are determined based on the saliency of geometric features and material stability. Points at weld edges, gusset plate corners, and connections of main load-bearing members are selected as feature points, and the screening results are stored as a set of feature point coordinates.

[0034] Optimize optical flow tracing algorithm execution First, Gaussian and Laplace pyramids are constructed, with the pyramid levels determined based on the degree of geometric asymmetry and natural frequencies. Then, consistency constraints are applied to the feature point trajectories, and anomaly trajectories are eliminated using a trajectory fitting algorithm. During trajectory tracking, the velocity and acceleration of feature points are calculated in real time; trajectory points exceeding the abrupt change threshold are identified as anomalies and removed.

[0035] (iv) Implementation of S4 vibration displacement calculation and twin fusion Establishing coordinate mapping relationships Based on the calibration parameters of the visual acquisition device, a transformation matrix between pixel coordinates and physical coordinates is established. The parameters of the transformation matrix are determined by the device's installation position, angle, and focal length. Matrix operations are used to convert the pixel coordinates of feature points into physical coordinates.

[0036] Deep learning model solution The deep learning model employs a combined architecture of Transformer and Long Short-Term Memory (LSTM) networks, with hyperparameters adjusted using the NRBO optimization algorithm during training. The training dataset is constructed from historical monitoring data of structural vibrations, and the training objective is to minimize the solution error. The model takes the trajectory data of feature points as input and outputs the vibration parameters and displacement data for each monitoring point.

[0037] Digital twin model fusion A digital twin model is constructed using point cloud scanning and BIM technology, incorporating the structure's geometric parameters, material properties, and dynamic coupling effect data. The calculated vibration and displacement data are transmitted to the digital twin model, and the virtual structure's state is updated synchronously to correct deviations caused by the mast antenna whiplash effect.

[0038] (V) Implementation of S5 multi-source verification and structural status linkage handling Multi-source data cross-validation Stress data from strain gauges and vibration acceleration data from accelerometers are collected and correlated with machine vision calculations. The hierarchical analysis algorithm is used to determine the verification weights of the three types of data, and the weighted fusion monitoring results are calculated to reduce the systematic error of a single monitoring method.

[0039] Structural status linkage handling The digital twin model generates dynamic response trigger conditions based on the structure's safety threshold. When monitoring data reaches these conditions, audible and visual alerts are activated, and data is pushed to multiple devices. Simultaneously, the visual acquisition device is triggered to perform encrypted sampling, and the digital twin model is used to perform elastoplastic time-history simulation. When monitoring data continuously exceeds limits and shows a damage evolution trend, suggestions for adjusting construction techniques and structural reinforcement schemes are generated.

[0040] Advantages compared to existing technologies Solving the challenges of monitoring complex structures Existing technologies employ single-resolution optical flow algorithms, which are prone to trajectory loss due to structural geometric asymmetry and abrupt changes in stiffness. This method optimizes the optical flow algorithm through a multi-scale pyramid, imposing trajectory consistency constraints to ensure continuous trajectory tracking. Simultaneously, a digital twin model is used to correct for mast-whiplash effect biases, improving the monitoring accuracy of specific locations.

[0041] Enhance multi-condition adaptability Existing technologies mostly focus on single seismic conditions and do not consider the coupling effect of wind load and temperature deformation. This method sets up a seismic condition adaptation module to adjust the model feature extraction weights for different seismic conditions. Combined with a scene-based adaptive module for image preprocessing, it can cope with environmental interference such as dust, light, rain and snow, and adapt to complex service environments.

[0042] Improve the reliability of monitoring data Existing technologies rely on a single monitoring method, resulting in significant data errors. This method reduces system errors by cross-validating multi-source data from machine vision, strain gauges, and accelerometers, combined with an error compensation algorithm. An automatic parameter adjustment mechanism across all stages adjusts parameters based on vibration intensity and environmental disturbances, ensuring stable operation of the monitoring system.

[0043] Achieve intelligent management and control throughout the entire lifecycle Existing technologies lack dynamic linkage between physical structures and virtual models, resulting in delayed responses. This method achieves seamless integration of monitoring data with construction adjustments and emergency response through bidirectional synchronous mapping of digital twin models. It is applicable to all stages of construction and operation and maintenance, providing data support for structural health assessment.

[0044] IV. Summary of Core Methodological Innovations Optimize the optical flow tracing algorithm: Solve the problem of trajectory loss in complex structures by constructing multi-scale pyramids and trajectory consistency constraints.

[0045] Multi-source data fusion verification: Combining machine vision, strain gauge, and accelerometer data, the reliability of monitoring results is improved through weighted fusion.

[0046] Digital twin bidirectional linkage: Real-time synchronization between physical structure and virtual model, correction of whiplash effect deviation, and support for intelligent assessment of structural status.

[0047] Automatic parameter control throughout the entire process: Parameters are dynamically adjusted based on vibration intensity and environmental disturbances to ensure stable operation of the monitoring system in complex scenarios.

[0048] Example Description I. Basic Experimental Conditions Experimental subjects A lattice-type cylindrical all-steel structure TV tower was selected, with the main body made of Q345 steel. The structure contains multiple open sections and exhibits abrupt changes in vertical stiffness. The top mast antenna has a dynamic coupling effect with the main structure, which conforms to the typical characteristics of tall steel structures.

[0049] Experimental equipment configuration The visual acquisition equipment uses industrial cameras, with parameters determined based on the structural vibration frequency and monitoring accuracy; the auxiliary monitoring equipment uses total stations, strain gauges, and triaxial accelerometers, with parameters determined based on monitoring performance requirements; the data processing equipment uses industrial control computers equipped with deep learning models and digital twin platforms.

[0050] II. Experimental Implementation Steps Example 1: Optical Flow Algorithm Optimization and Digital Twin Fusion Monitoring Step S1 involves dividing the monitoring layer and deploying visual acquisition devices to collect reference point coordinate data and establish a global coordinate system. Step S2 involves image preprocessing, using adaptive filtering and grayscale stretching to process the image sequence. Step S3 involves feature point extraction and trajectory tracking, employing an optimized optical flow algorithm to ensure trajectory continuity. Step S4 involves displacement calculation and digital twin fusion, using a deep learning model to calculate the data and transmitting it to the digital twin model to correct for whiplash effect bias.

[0051] Example 2: Multi-source verification and structural state linkage processing Based on Example 1, strain gauges and accelerometers were deployed to collect data, which was then cross-validated with machine vision results. Verification weights were assigned using an analytic hierarchy process (AHP) and the data was fused to verify reliability. Dynamic response trigger conditions were set; when the monitored data reached these conditions, a coordinated response process was initiated, generating construction adjustment suggestions.

[0052] Example 3: Seismic Condition Adaptation and Full-Process Parameter Control Based on Example 1, the seismic condition adaptation module is activated to adjust the model feature extraction weights for different seismic conditions. Through the automatic parameter control module across all stages, the sampling frequency and exposure time are adjusted according to vibration intensity, and the preprocessing algorithm parameters are optimized based on environmental interference to ensure stable monitoring accuracy.

[0053] III. Analysis of Experimental Results Example 1 achieves seamless trajectory tracking and displacement calculation accuracy that meets engineering requirements by optimizing the optical flow algorithm and fusing it with digital twins. Example 2 improves the reliability of monitoring results and significantly reduces processing delays by using multi-source verification and coordinated handling. Example 3 adapts to multiple disaster coupling scenarios by adapting to seismic conditions and adjusting parameters, and maintains stable monitoring accuracy.

[0054] Traditional monitoring methods rely on single sensors and threshold judgments, which suffer from problems such as trajectory loss, large data deviations, and response lags, failing to meet the monitoring needs of complex structures. This method, through technological innovation, comprehensively addresses the shortcomings of existing technologies and possesses significant value for industrial applications.

[0055] The embodiments described herein have been described in sufficient detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be used and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. Therefore, the detailed description should not be construed as limiting, and the scope of the various embodiments is defined only by the appended claims and the full scope of their equivalents.

Claims

1. A method for monitoring vibration and identifying displacement of a high-rise steel structure based on machine vision, characterized in that, Comprising the following steps: S1 visual acquisition and reference calibration: along the height direction and the circumference of the high-rise steel structure, according to the mechanical properties of the vertical stiffness mutation area and the dynamic response sensitive parts of the structure geometrically asymmetric distribution characteristics, visual acquisition equipment is arranged; A three-dimensional coordinate system is established based on the structure installation reference surface, fixed reference monitoring points are arranged at the symmetric area of the tower foot and the root of the mast antenna, three-dimensional coordinate data of the reference points are synchronously collected, the collection range completely covers the node concentrated area and the key stress part of the structure transparent area, real-time continuous image sequences of the structure surface are captured, the resolution and sampling frequency of the visual acquisition equipment are determined according to the inherent frequency and vibration response characteristics of the structure, and after the equipment is installed, calibration is completed through the reference point coordinates; S2 image preprocessing and feature enhancement: sequentially execute noise suppression, light balance and spatial registration operations on the collected image sequences; signal interference caused by device jitter and environmental dust is eliminated through an adaptive filtering algorithm, brightness deviation caused by light changes is corrected using a gray scale stretching technique, accurate alignment of continuous frame images is completed based on three-dimensional coordinates of the reference monitoring points, gray scale gradient and geometric recognition of mechanically stable parts such as weld edge node plate corner points are strengthened, filter kernel parameters of the adaptive filtering algorithm are dynamically adjusted according to the intensity of environmental interference, and the gray scale range of the gray scale stretching is determined according to the reflection characteristics of the structure surface material; S3 feature point extraction and trajectory tracking: in the preprocessed image, point positions with significant geometric features and stable material characteristics are selected, which meet the requirements of structural mechanics stability and cover weld edge node plate corner points and main stress member connection points; an optimized optical flow tracking algorithm is used to track the motion trajectory of the feature points in the continuous image frames, abnormal trajectory points generated by non-structural disturbance are automatically removed, and the determination of abnormal trajectory points is based on the mutation threshold of the feature point motion speed and acceleration, which is determined by the structural dynamic response characteristics; S4 vibration displacement solving and twin fusion: based on the feature point motion trajectory, combined with the geometric size parameters, material mechanics properties and visual acquisition equipment calibration data of the high-rise steel structure, a mapping relationship between pixel coordinates and physical coordinates is established; vibration parameters and displacement data of each monitoring point of the structure are solved through a deep learning model, and the solving results are transmitted to the digital twin model of the high-rise steel structure in real time, realizing real-time synchronous mapping of the physical structure state and the virtual model; S5 multi-source verification and structure state linkage disposal: structure stress data and vibration acceleration data are collected through the deployed strain gauges and accelerometers, and are cross-verified with the machine vision solving results; According to the monitored data after verification, combined with the structural modal analysis results, material mechanics properties and seismic response law, the corresponding disposal process is automatically started, providing data support for construction process adjustment and disaster emergency disposal, and the cross-verification is executed through data correlation analysis algorithm, and the correlation threshold is determined by the structural monitoring accuracy requirement.

2. The method of claim 1, wherein the method further comprises: The optimized optical flow tracking algorithm in step S3 is executed according to the following steps: The pre-processed image is first down-sampled at multiple scales to construct a Gaussian pyramid, and then a Laplacian pyramid is generated by difference between adjacent scale images, the pyramid level is determined by the geometric asymmetry degree of the high-rise steel structure and the inherent frequency obtained by the multiple Ritz vector method, the down-sampling factor of the Gaussian pyramid is determined by the image resolution and the structural detail feature size, and the difference operation of the Laplacian pyramid is calculated according to the pixel difference of adjacent scale images; Combined with the periodicity law of structural vibration and the vibration mode decomposition result, a consistency constraint is imposed on the feature point motion trajectory, when the trajectory deviates from the periodic trend, based on the trajectory data of the previous period and the vibration mode parameters of modal analysis, local re-tracking is started, and the pixel coordinate deviation of the feature point is corrected, the consistency constraint is executed through the trajectory fitting algorithm, the fitting function is determined by the periodic characteristics of structural vibration, and the range of local re-tracking is determined by the trajectory deviation degree.

3. The method of claim 2, wherein the method further comprises: The deep learning model in step S4 is constructed and applied as follows: first, the spatial correlation features of the feature point motion trajectory are extracted through the multi-head attention mechanism of the Transformer model, focusing on capturing the trajectory difference caused by the stiffness difference between the structural node region and the mid-span of the member; the attention weight of the multi-head attention mechanism is determined by the spatial distance and correlation of the feature point trajectory; Then, the time sequence dynamic features of the trajectory are captured through the gating unit of the long short-term memory network, and the periodic law of the vibration signal and the mutation feature recognition under the action of the earthquake are strengthened; During the model training process, the meta-heuristic NRBO optimization algorithm is introduced to automatically adjust the network hyperparameters, including the number of attention heads, the number of hidden layer neurons, and the learning rate; the hyperparameter adjustment is based on the spatial distribution characteristics of the time sequence continuity displacement data of the high-rise steel structure vibration signal and the seismic vulnerability curve data, automatically matching the monitoring needs of members of different heights and different stress types.

4. The method of claim 3, wherein the method further comprises: The digital twin model in step S4 is constructed and fused as follows: the three-dimensional geometric data of the high-rise steel structure is obtained through point cloud scanning technology, and the digital twin model is reconstructed and generated by combining BIM technology and elastoplastic time history analysis results, the model includes the geometric parameters, material properties, construction stage information, design safety threshold, and seismic response characteristic data of the structure, while incorporating the dynamic coupling effect data of the mast antenna and the main structure; The fusion of the digital twin model and the monitoring data adopts a bidirectional real-time synchronous mapping method, which receives the calculation data of step S4, adjusts the synchronous period according to the structural stress change and seismic response characteristics of the construction stage, and updates the vibration state and displacement distribution of the virtual structure in real time, focusing on correcting the monitoring deviation caused by the whip effect of the mast antenna.

5. The method of claim 4, wherein the method further comprises: The multi-source verification in step S5 is performed as follows: First, according to the monitoring accuracy characteristics, response speed, and environmental adaptability of machine vision strain gauges and accelerometers, dynamically allocate the verification weights of the three types of data, and automatically adjust the weight allocation according to the measurement deviation law of each monitoring method under the coupling action of seismic wind load and temperature deformation; When a certain type of monitoring data deviates due to environmental interference, call the historical monitoring data and structural elastoplastic time history analysis data stored in the digital twin model for error compensation; The stress data and the vibration acceleration data are correlated and analyzed with displacement data calculated by machine vision, and reliability of the monitoring result is verified by combining with earthquake vulnerability curve data, so as to reduce system error of a single monitoring mode through a data fusion algorithm.

6. The method of claim 5, wherein the method further comprises: The image preprocessing of step S2 sets a scene adaptive processing module, which deals with different environmental interferences according to the following steps: When encountering dust interference, morphological operation is used to remove floating dust noise, and then image restoration algorithm is used to repair pixel information of the feature point area, and the repair process adjusts algorithm parameters according to the gray gradient distribution law of the feature point and the material characteristics of the structure surface; When encountering light changes, adaptive threshold segmentation is performed to adjust the gray scale range, and then gray scale equalization technology is used to correct brightness deviation, so as to ensure that the gray scale contrast between the feature point and the background meets the recognition requirements; When encountering rain and snow interference, rain and snow pixels are identified by pixel gray difference analysis, and then filtering algorithm is used to remove noise, and the size of the filtering kernel is adjusted according to the pixel size of the rain and snow particles and the geometric characteristics of the structure surface.

7. The method of claim 6, wherein the method further comprises: The structural state linkage treatment of S5 is performed according to the following steps: According to the modal analysis results of the high-rise steel structure, the material mechanics characteristics and the seismic response law, the structural dynamic response trigger condition is automatically generated from the digital twin model, and the condition is adjusted in real time according to the stress state of the structure construction progress and the coupling strength of multiple disasters; When the monitoring data reaches the condition, the system automatically starts the treatment process, synchronously pushes the multi-end data to the related technical and management posts through sound and light prompts, and triggers the visual acquisition equipment in the local area to encrypt sampling and the digital twin model to carry out elastic-plastic time history simulation deduction, so as to verify the authenticity of the structural dynamic response; When the monitoring data continuously exceeds the condition and the simulation deduction shows that the structure has a damage evolution trend, the system automatically generates construction process adjustment suggestions and structure reinforcement schemes.

8. The method of claim 7, wherein the method further comprises: The visual acquisition equipment deployment of step S1 is performed according to the following steps: Along the height direction of the high-rise steel structure, the monitoring layers are divided according to the node concentration degree and vertical stiffness mutation of the main force member, and the spacing between the monitoring layers is determined according to the attenuation law of the structural dynamic response and the modal analysis results of the multiple Ritz vectors; Each layer is circumferentially arranged according to the symmetry principle, the equipment installation position is avoided to be in the structure shielding area, the installation angle is adjusted according to the inclination angle of the feature point plane, and the lens optical axis is perpendicular to the feature point plane; during the construction stage, the equipment position and the acquisition range are adjusted according to the structure assembly progress and the change of the vibration sensitive area, and the monitoring coverage of the mast antenna root and the transparent area is emphasized.

9. The method of claim 8, wherein the method further comprises: The full-link parameter automatic regulation step is also included, which is implemented as follows: State monitoring modules are respectively set in the visual acquisition equipment data processing unit and the digital twin platform to collect equipment working parameters, algorithm running parameters and model synchronization parameters in real time; The control unit receives the environmental interference data of step S6, the tracking effect of step S3 and the calculation accuracy feedback of step S4, and automatically adjusts the parameters of each link. When the vibration intensity increases, the sampling frequency and feature point extraction density are improved, and the exposure time is adjusted to reduce motion blur. The sampling frequency adjustment is consistent with the natural frequency obtained from the structural modal analysis. When the environmental disturbance is severe, the pre-processing algorithm parameters are optimized to enhance the anti-interference ability.

10. The method of claim 9, wherein the method further comprises: The vibration displacement calculation of step S4 also includes a seismic working condition adaptation module, which performs the following steps: For different seismic working conditions such as far-field long-period earthquakes and near-field pulse earthquakes, the feature extraction weights of the deep learning model are adjusted in combination with the dynamic response characteristics and mode decomposition results of the high-rise steel structure to strengthen the calculation logic of the coupling relationship between long-period vibration signals and displacement. During the calculation process, the digital twin model simulates the structural deformation trend under seismic action, and the calculation results are supplemented and verified in combination with the seismic vulnerability curve. At the same time, the structure seismic vulnerability analysis data is introduced to correct the displacement calculation deviation caused by the whip effect of the mast antenna.

Images