A Method and System for Synchronous Monitoring of Bridge Structural Displacement Based on Wireless Intelligent Vision

Abstract

This invention discloses a method and system for synchronous monitoring of bridge structural displacement based on wireless intelligent vision, belonging to the field of bridge health monitoring technology. The method involves acquiring images of the bridge monitoring area and performing format standardization processing; synchronizing the clock and frame acquisition timing of multiple monitoring nodes, achieving microsecond-level synchronization through linear drift compensation, hardware-triggered synchronization, and frame buffering technology; calculating sub-pixel displacement data using either target-independent or improved marker tracking modes, the former based on frequency domain phase correlation analysis, and the latter using a constrained attitude decomposition algorithm to improve accuracy; performing localized visual calculations on the displacement data and outputting displacement time-series data; and wirelessly transmitting the data to achieve real-time visualization and data interaction with downstream systems. The system includes modules for image acquisition, node synchronization, displacement calculation, local calculation, wireless transmission, and data interaction. This invention achieves non-contact, high-precision displacement monitoring, improves multi-node synchronization and data real-time performance, and is adaptable to various inspection scenarios.

Description

Technical Field

[0001] This invention relates to the field of bridge health monitoring technology, specifically to a method and system for synchronous monitoring of bridge structural displacement based on wireless intelligent vision. Background Technology

[0002] In the field of bridge structural health monitoring, displacement monitoring is one of the core indicators for assessing structural safety. Traditional monitoring methods mostly rely on contact sensors, such as wire displacement gauges and strain gauges. These devices need to be directly installed on the surface of the bridge structure, which is not only difficult to install and costly to maintain, but also susceptible to the effects of ambient temperature, humidity, and structural vibration, resulting in limited measurement accuracy and long-term stability. At the same time, existing monitoring systems mostly use wired transmission, which involves complex wiring and long deployment cycles, making it difficult to meet the wide-area, multi-point monitoring needs of large bridges, especially elevated railway bridges, and unable to meet the application scenarios of rapid inspection and unattended operation.

[0003] With the development of visual measurement and wireless communication technologies, non-contact monitoring solutions have gradually become a research hotspot in the industry. Machine vision-based displacement monitoring methods acquire image sequences of bridge structures and calculate displacement using image matching and feature tracking techniques. This eliminates the need for direct contact with the structure, offering advantages such as flexible deployment and strong anti-interference capabilities. The introduction of wireless communication technology further reduces system deployment costs and improves the scalability of the monitoring network. However, existing visual monitoring solutions still have significant shortcomings in terms of multi-node synchronization, real-time performance, and accuracy stability: clock drift and frame rate fluctuations between multiple nodes can easily lead to inconsistencies in time-series data, affecting the reliability of subsequent processing such as modal analysis; visual algorithms often rely on offline post-processing, making it difficult to achieve real-time output of displacement data; while some solutions use marker point tracking to improve accuracy, these marker points are easily affected by environmental occlusion and changes in lighting, failing to simultaneously meet the dual requirements of long-term monitoring and rapid inspection.

[0004] Chinese patent (publication number: CN113465557A) proposes a displacement monitoring scheme for elevated bridges based on multi-point signal transceivers. By deploying monitoring base points at the bottom of the bridge piers, the scheme simultaneously monitors the displacement of multiple piers using signal transceivers, and decomposes the displacement into horizontal and vertical components for integrated analysis, generating monitoring curves for staff verification. This scheme effectively improves the coverage and intuitiveness of displacement monitoring through multi-point deployment and component decomposition, solving the problems of single-point monitoring and abstract results associated with traditional contact sensors. However, this scheme still has significant shortcomings: the signal transceivers it relies on are essentially quasi-contact measurement devices, requiring the deployment of dedicated monitoring base points and signal devices on the bridge piers, resulting in high deployment costs and construction difficulties; the monitoring process relies on fixed reference points, making it susceptible to the influence of the reference points' own settlement or displacement, leading to a decrease in measurement accuracy over time; furthermore, the scheme does not incorporate visual measurement and edge computing technologies, failing to achieve non-contact, real-time displacement calculation, and is ill-suited for long-term unattended monitoring in complex environments.

[0005] Chinese patent (publication number: CN113466846A) proposes a phased, refined monitoring scheme for high-speed railway bridge offset monitoring. This scheme monitors the horizontal and vertical offsets of the bridge piers and beams in real time during three key stages: jet grouting pile construction, beam relocation construction, and train passage construction. It outputs multiple sets of offset data and, through refined processing, obtains a complete sequence containing the correspondence between time and displacement, plotting the offset change process at each stage. This scheme, through phased monitoring and refined data processing, effectively improves the targeting and data completeness of the monitoring, solving the problems of reference point dependence and missing monitoring during the construction period in previous schemes. However, this scheme still has core shortcomings: its monitoring methods still rely mainly on traditional sensors, without adopting non-contact visual measurement technology, resulting in high deployment and maintenance costs; multi-stage monitoring depends on switching between different sensors and data integration, leading to high system complexity and potential data gaps and inconsistencies in timing; furthermore, the scheme does not incorporate wireless synchronization and edge computing technologies, failing to achieve microsecond-level synchronization and localized real-time calculation across multiple nodes, making it difficult to meet the wide-area, real-time, and unattended monitoring requirements of large railway bridges.

[0006] Therefore, there is an urgent need for a non-contact, high-precision, real-time, and scalable bridge structure displacement synchronous monitoring solution to address the problems of existing technologies in terms of deployment cost, synchronization accuracy, real-time performance, and environmental adaptability. Summary of the Invention

[0007] To address the aforementioned technical problems, this application discloses a method and system for synchronous monitoring of bridge structure displacement based on wireless intelligent vision. The method for synchronous monitoring of bridge structure displacement based on wireless intelligent vision specifically includes:

[0008] Image data of the bridge structure monitoring area is collected, and the image data is processed to standardize the format.

[0009] Clock synchronization and frame acquisition timing synchronization are performed on multiple monitoring nodes to ensure the timing consistency of image data acquired by each node.

[0010] The displacement of the standardized image data is calculated by using a target-independent tracking mode or an improved marker tracking mode to obtain sub-pixel displacement data of the bridge structure.

[0011] The subpixel displacement data is processed using localized visual computation to output displacement time series data of the bridge structure.

[0012] The displacement time series data is wirelessly transmitted to enable real-time visualization of the displacement data and data interaction with downstream systems.

[0013] Preferably, the clock synchronization and frame acquisition timing synchronization processing for multiple monitoring nodes includes:

[0014] Perform two clock synchronization operations on each monitoring node to obtain the synchronization timestamp and the corresponding clock deviation value, and eliminate the clock deviation between each monitoring node through a linear drift compensation algorithm.

[0015] The global shutter sensor is connected to the interrupt signal of the wireless node via hardware triggering to achieve hardware synchronization of frame acquisition by each monitoring node.

[0016] By using frame buffering technology to stabilize the image acquisition frame rate of each monitoring node, and by combining dynamic window adjustment to optimize frequency domain calculation efficiency, and by periodically clearing the cache and garbage collection, microsecond-level synchronization of each monitoring node is achieved.

[0017] Preferably, the step of eliminating clock skew between monitoring nodes using a linear drift compensation algorithm includes:

[0018] Let the timestamps for the two clock synchronizations be... The corresponding clock offset value is The linear interpolation method is used to obtain the value at any time. Clock skew compensation value ;

[0019] According to the clock skew compensation value Real-time compensation is performed on the local clocks of each monitoring node to eliminate timing errors caused by clock drift. The formula is: , This refers to any time when clock skew compensation is performed.

[0020] Preferably, the displacement calculation of the standardized image data using the target-independent tracking mode includes:

[0021] The first frame image before the load is applied is selected as the reference frame, and the image acquired during the monitoring process is used as the current frame to determine the region of interest for monitoring the two frames.

[0022] Two-dimensional fast Fourier transforms are performed on the regions of interest of the reference frame and the current frame respectively, and the phase correlation matrix is ​​calculated through frequency domain phase correlation analysis;

[0023] The phase correlation matrix is ​​subjected to subpixel interpolation using the peak centroid method to calculate the subpixel displacement of the monitoring area.

[0024] Preferably, the phase correlation matrix is ​​calculated through frequency domain phase correlation analysis, and the formula is: ,in, The phase correlation matrix, The result of the two-dimensional Fast Fourier Transform of the region of interest in the reference frame. This is the conjugate result of the two-dimensional Fast Fourier Transform of the region of interest in the current frame. These are the frequency components in the horizontal and vertical directions, respectively.

[0025] Preferably, the step of performing displacement calculation on the standardized image data using an improved marker tracking mode includes:

[0026] Identify the coordinates of the marked corner points in the image and obtain the homography matrix by solving the linear least squares method;

[0027] Based on the motion characteristics of bridge vibration, out-of-plane motion is ignored and a constrained attitude decomposition algorithm is introduced to decompose the homography matrix and simplify the scale factor calculation process.

[0028] By combining the camera intrinsic parameter matrix, the actual displacement of the bridge structure is obtained by solving the column vector of the homography matrix, and a quality factor is assigned to each frame of image to automatically identify and remove outliers in the displacement calculation.

[0029] Preferably, the actual displacement of the bridge structure is calculated by combining the camera intrinsic parameter matrix and solving the column vectors of the homography matrix, using the following formula: ,in, The column vectors of the homography matrix. For the camera intrinsic parameter matrix, Let be the column vector of the camera pose rotation matrix. Let be the camera pose translation vector. is the scale factor.

[0030] Preferably, the step of performing localized visual computation processing on the subpixel displacement data to output displacement time series data of the bridge structure includes:

[0031] Visual computation operations such as fast Fourier transform, phase correlation analysis, and camera pose estimation are performed locally on a low-power processor.

[0032] Based on the actual needs of bridge monitoring, a real-time calculation mode or a near-real-time calculation mode is selected to perform time-series processing on subpixel displacement data and output continuous displacement time series data. The near-real-time calculation mode improves the data output frame rate through frame buffering technology.

[0033] Preferably, wireless transmission of the displacement time series data includes:

[0034] The displacement time series data is packetized and encoded, and the encoded data packets are transmitted between nodes and sent to the terminal via a wireless communication protocol.

[0035] During wireless transmission, a low-jitter communication interface is used to achieve synchronous transmission and reception of image acquisition data and displacement calculation data, ensuring the integrity and timing of the transmitted data.

[0036] The bridge structure displacement synchronous monitoring system based on wireless intelligent vision includes:

[0037] The image acquisition module acquires image data of the bridge structure monitoring area and performs format standardization processing on the image data, including noise reduction, size normalization, and color space conversion.

[0038] The node synchronization module performs clock synchronization and frame acquisition timing synchronization processing on multiple monitoring nodes. It eliminates clock deviation between nodes through linear drift compensation and achieves microsecond-level synchronization of each monitoring node by combining hardware-triggered synchronization and frame buffering, thus ensuring the timing consistency of image data of each node.

[0039] The displacement calculation module uses a target-independent tracking mode or an improved marker tracking mode to perform displacement calculation on the standardized image data, and obtains the sub-pixel displacement data of the bridge structure through frequency domain phase correlation analysis or constrained attitude decomposition algorithm.

[0040] The local computing module performs localized visual computing on subpixel displacement data on a low-power processor. It selects real-time or near-real-time computing mode according to monitoring requirements and outputs time-series displacement data of the bridge structure after time-series processing of the subpixel displacement data.

[0041] The wireless transmission module performs packet encoding on displacement time series data, completes the inter-node transmission and terminal sending of data packets through a wireless communication protocol, and realizes synchronous transmission and reception of collected and processed data through a low-jitter communication interface to ensure the integrity and timing of transmitted data.

[0042] The data interaction module receives displacement time series data sent by the wireless transmission module, realizes real-time visualization of displacement data, and completes data interaction with the downstream structural health assessment platform.

[0043] Compared with the prior art, the technical solution of this application has the following technical effects:

[0044] This invention achieves full adaptability to bridge displacement monitoring scenarios through a dual-mode displacement tracking design. The target-independent tracking mode does not require the deployment of physical markers and can quickly complete displacement screening of key areas of the bridge. The improved marker tracking mode relies on a constrained attitude decomposition algorithm to effectively avoid out-of-plane motion interference and improve the stability of displacement calculation. The two modes can be flexibly switched to meet the core needs of different monitoring scenarios.

[0045] The multi-node synchronization mechanism constructed in this invention achieves time-series uniformity of monitoring nodes through three layers: clock, hardware, and software. Linear drift compensation eliminates clock deviations between nodes, while hardware triggering and frame buffering technologies ensure the synchronization of frame acquisition, enabling multi-node data acquisition to have high temporal consistency and providing a reliable data foundation for bridge structural modal analysis.

[0046] This invention completes the localized processing of visual data based on an edge computing architecture. Core calculations such as Fourier transform and phase correlation analysis can be performed at the monitoring node, and displacement time series data can be directly output. The localized processing mode eliminates the need for long-distance transmission of massive image data, significantly improves the output efficiency of displacement data, and realizes real-time presentation of monitoring results.

[0047] This invention uses wireless communication technology to construct a monitoring network. Combined with low-power hardware selection and lightweight algorithm design, it greatly improves the system's deployment flexibility and environmental adaptability. The monitoring nodes can achieve wireless networking over a wide area without complex wiring, making it suitable for long-term unattended monitoring scenarios in the field. At the same time, it ensures the integrity and continuity of displacement monitoring data transmission.

[0048] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0049] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0051] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows:

[0052] Figure 1 A schematic diagram of the overall process of a bridge structure displacement synchronous monitoring method based on wireless intelligent vision;

[0053] Figure 2 A detailed flowchart illustrating the image acquisition and format standardization process for the bridge structure monitoring area;

[0054] Figure 3 A schematic diagram illustrating the implementation process of clock synchronization and frame acquisition timing synchronization for multiple monitoring nodes;

[0055] Figure 4 A schematic diagram of the dual-mode subpixel displacement calculation method and data fusion process for bridge structures;

[0056] Figure 5 A schematic diagram of the overall architecture of a bridge structure displacement synchronous monitoring system based on wireless intelligent vision. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0058] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0059] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0060] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0061] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0062] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0063] Example 1

[0064] This embodiment mainly describes a bridge displacement synchronous monitoring method based on wireless intelligent vision, such as... Figure 1 As shown, it specifically includes:

[0065] Image data of the bridge structure monitoring area is collected, and the image data is processed to standardize the format.

[0066] Clock synchronization and frame acquisition timing synchronization are performed on multiple monitoring nodes to ensure the timing consistency of image data acquired by each node.

[0067] The displacement of the standardized image data is calculated by using a target-independent tracking mode or an improved marker tracking mode to obtain sub-pixel displacement data of the bridge structure.

[0068] The subpixel displacement data is processed using localized visual computation to output displacement time series data of the bridge structure.

[0069] The displacement time series data is wirelessly transmitted to enable real-time visualization of the displacement data and data interaction with downstream systems.

[0070] Furthermore, such as Figure 2 As shown, image data of the bridge structure monitoring area is collected, and the image data is standardized in format. The specific execution process is as follows: Based on the mechanical characteristics of the bridge structure, key monitoring sections are delineated as image acquisition areas. Image acquisition terminals are deployed in each monitoring area. The terminals are equipped with global shutter sensors or rolling shutter sensors, and the pixel array size of the sensors is [missing information]. The sensor's exposure time is configured via the I2C bus during data acquisition. Gain coefficient With line frequency parameters, the sensor operates at a preset frame rate. Acquire consecutive image frames, and output the raw image data in Bayer RAW format, denoted as... ,in , For pixel coordinates, This refers to the Bayer channel. Demosaic processing is performed on the original RAW image, and the color image is restored using bilinear interpolation. The formula is:

[0071]

[0072]

[0073]

[0074] in , , These represent the pixel values ​​for the red, green, and blue channels, respectively. After demosaicing, Gaussian filtering is performed for noise reduction, with the filter kernel being... Gaussian kernel The formula is: , in The standard deviation of the Gaussian kernel. This is the denoised color image. Size normalization is performed, and the image is scaled to a preset resolution using bicubic interpolation. The formula is: , ,in , These are the normalized pixel coordinates. , These are the pixel coordinates of the original image. This is the bicubic interpolation weighting function. It performs a color space conversion, converting the RGB image to a grayscale image, using the following formula: Complete the standardized grayscale image format. As input data for subsequent displacement calculations, the data format is uniformly 8-bit unsigned integers, and the pixel value range is [missing value]. .

[0075] Furthermore, such as Figure 3 As shown, clock synchronization and frame acquisition timing synchronization are performed on multiple monitoring nodes to ensure the timing consistency of image data acquired by each node. The specific execution process is as follows: A multi-node clock synchronization model is constructed, which includes a master node and slave nodes. The master node broadcasts synchronization commands to all slave nodes through a wireless communication link. The synchronization process consists of four core steps: clock deviation calibration, clock drift compensation, hardware-triggered synchronization, and frame buffer frequency stabilization. First, two clock synchronization operations are performed, with the master node at time... Send the first synchronization frame, which contains the master node timestamp. Record the local timestamp after receiving from the node. And calculate the initial clock offset. The master node at time Send a second synchronization frame, which contains the master node timestamp. Record the local timestamp after receiving from the node. Calculate the secondary clock offset .

[0076] Based on two synchronization data sets, a clock drift model is established, whereby the drift coefficient of the slave node's local clock is set as follows. Then at any time Master node time With the local time of the slave node The relationship is: Substituting the data from the two synchronizations, we get: The drift coefficient is obtained by solving. With intercept : Thus, at any time Clock skew compensation value The formula is: Simultaneously, linear interpolation is used to correct the compensation value in real time, and the formula is: From the node according to The local clock is calibrated in real time to eliminate clock drift.

[0077] After clock synchronization is completed, hardware-triggered synchronization is performed, and the master node's wireless communication unit outputs an interrupt signal through the GPIO pin. All slave node image sensor trigger pins are connected to this interrupt signal; the sensor only responds when it receives the interrupt signal. The signal triggers frame acquisition, achieving hardware synchronization for frame acquisition. The pulse width of the trigger signal is [missing information]. This meets the sensor's trigger threshold requirements. To address the frame rate fluctuation issue, an SDRAM frame buffer model is introduced, with a frame buffer storage depth of [missing information]. Using a first-in, first-out (FIFO) storage strategy, the acquired image frames are written to the buffer sequentially, denoted as... When the buffer is full, the earliest written frame is overwritten.

[0078] Simultaneously, a frame rate stabilization model is constructed to calculate the current frame storage amount in the buffer. ,like If so, reduce the sensor's frame rate; To increase the sensor's frame rate, the frame rate adjustment formula is: The efficiency of frequency domain computation is optimized by adjusting the dynamic window. Let the FFT computation window size be... Adjust the window size based on the image texture complexity. The formula for calculation is: ;

[0079] The formula for adjusting window size is: ,in , A preset texture complexity threshold is set. Within a preset time interval. Internal cache cleanup and garbage collection are performed to release memory space occupied by invalid data. Through the above multi-stage synchronization model, microsecond-level timing synchronization of each monitoring node is achieved, ensuring the timing consistency of the acquired image data.

[0080] Furthermore, such as Figure 4 As shown, the target-independent tracking mode is used to perform displacement calculation on the standardized image data to obtain the sub-pixel displacement data of the bridge structure. The specific execution process is as follows: construct a target-independent displacement calculation model, which includes five core parts: ROI delineation module, frequency domain transformation module, phase correlation calculation module, sub-pixel interpolation module, and displacement calculation module.

[0081] Standardized grayscale image The monitoring area is defined as a region of interest (ROI), with the coordinates of its upper left corner as follows: Width is The height is The image of the ROI region is denoted as ,in ,satisfy , The ROI of the first frame image before load application is selected as the reference frame. The first data collected during the monitoring process The ROI of the frame image is used as the current frame. Perform a two-dimensional Fast Fourier Transform (FFT) on both the reference frame and the current frame to convert the spatial domain data into frequency domain data. The FFT calculation formula is as follows: ,in For spatial domain image pixel values, For frequency domain representation, , The image size is used to derive the frequency domain representation of the reference frame. Frequency domain representation of the current frame ,in , Use frequency domain coordinates. Calculate the frequency domain cross-power spectrum. The formula is: ,in This is the complex conjugate of the current frame's frequency domain representation. Normalizing the cross-power spectrum yields the phase correlation matrix. The formula is: Performing an inverse fast Fourier transform (IFFT) on the phase correlation matrix converts the frequency domain data back to the spatial domain, yielding the correlation peak image. The formula for calculating IFFT is: In the relevant peak image, the peak position The integer pixel displacement between the reference frame and the current frame, i.e. To achieve sub-pixel accuracy, the peak centroid method is used for sub-pixel interpolation of relevant peaks. First, the peak neighborhood window is determined, with a window size of [missing information]. The pixel value in the neighborhood is , Calculate the centroid coordinates within the neighborhood. The formula is:

[0082]

[0083]

[0084] The subpixel displacement is By combining the camera calibration parameters, the pixel displacement is converted into actual physical displacement, and the camera's pixel equivalent is... That is, the physical distance corresponding to a unit pixel, the actual displacement. , The calculation formula is: This physical displacement is the bridge structure at the [missing value]. The subpixel displacement data at the frame acquisition time is denoted as .

[0085] Furthermore, such as Figure 4 As shown, an improved marker-tracking model is used to calculate the displacement of standardized image data, obtaining sub-pixel displacement data of the bridge structure. The specific execution process is as follows: An improved marker-tracking displacement calculation model is constructed, comprising five core parts: a marker corner detection module, a homography matrix solving module, a constraint attitude decomposition module, a displacement calculation module, and an outlier removal module. First, AprilTag markers are deployed in the bridge monitoring area, with the world coordinates of the markers being... ,in This indicates that the marker is located in a two-dimensional plane. In the standardized grayscale image... In this study, an edge detection algorithm is used to extract the edge features of the markers. The edge detection employs the Canny operator and consists of four steps: Gaussian filtering, gradient calculation, non-maximum suppression, and double threshold detection. The gradient magnitude is... With direction The calculation formula is: , , The four corner points of the marker are identified by edge feature matching, and the image coordinates of the corner points are as follows: , The corresponding world coordinates are Based on the image coordinates and world coordinates of the corner points, the homography matrix is ​​solved using the linear least squares method. The homography matrix is A homogeneous matrix that satisfies Expanded into a system of linear equations:

[0086]

[0087] Rewrite the system of equations in matrix form. ,in for The coefficient matrix, for A 9-dimensional column vector composed of elements is solved using singular value decomposition (SVD). The formula is: ,Pick The last column of the matrix is ​​used as The solution is used to reconstruct the homography matrix. and normalization processing Based on the motion characteristics of bridge vibration, neglecting out-of-plane motion, a constrained attitude decomposition algorithm is introduced, and the homography matrix can be decomposed into... ,in The camera intrinsic parameter matrix is ​​in the following form: , , The pixel equivalent of the camera's focal length. , The coordinates of the principal point are used. Since out-of-plane motion is ignored, the third column of the rotation matrix... ,therefore and Satisfying the orthogonality condition , .make , , scale factor The calculation formula is: ,but , The calculation formula is: Translation vector The calculation formula is: Translation vector The first two elements , This refers to the planar displacement of the camera relative to the marker, corresponding to the actual displacement of the bridge structure. The formula is: To eliminate outliers, a quality factor assessment model is constructed, and the quality factor... The formula for calculating the confidence level of corner detection based on the pixel contrast of the marked region is as follows: ,in These are the weighting coefficients. The contrast of the marked region is calculated using the following formula: , A grayscale image of the marked region. This represents the confidence score for corner detection, output by the corner detection algorithm. If... ( If the preset threshold is used, the displacement calculation result of that frame is determined to be an outlier and discarded; otherwise, it will be... As subpixel displacement data of the bridge structure, denoted as .

[0088] The subpixel displacement data undergoes localized visual computation processing to output the displacement time series data of the bridge structure. The specific execution flow is as follows: a localized edge computing model is constructed and deployed on a low-power embedded processor. This model comprises four core parts: a computation task scheduling module, a core algorithm computation module, a data time series processing module, and an output control module. The computation task scheduling module, based on the processor's computing power, uses a priority scheduling algorithm to sort computation tasks, with priority... The calculation formula is: ,in , , These are the weight coefficients for each task. The deadline for the task. For the amount of task data, The computational complexity of the task is determined by the core algorithm's computational module, which performs FFT calculations, phase correlation analysis, homography matrix solving, constrained pose decomposition, and camera pose estimation locally on the processor. All calculations are based on the aforementioned formulas. Fixed-point arithmetic optimization techniques are employed during the calculation process, converting floating-point operations to fixed-point operations to reduce computational latency and power consumption. For the acquired continuous sub-pixel displacement data... The data timing processing module performs timing processing, first by resampling the data and then using linear interpolation to unify the displacement data to a preset time step. The resampled displacement data is denoted as , The timestamp is used, followed by data smoothing using a sliding window filtering algorithm with a window size of [value missing]. The formula is: Then, detrending processing is performed, and the trend term of the displacement data is fitted using the least squares method. The detrended displacement data is The output control module selects between real-time and near-real-time calculation modes based on monitoring requirements. In real-time mode, the corresponding displacement data is output immediately after each frame of displacement calculation is completed. In near-real-time mode, the displacement data is buffered through the SDRAM frame buffer module. Frame offset data, batch output, output frame rate The calculation formula is: ,in This refers to the cache duration.

[0089] The processed displacement data is sorted by timestamp to form a continuous displacement time series data. The data format is a preset structured data format, which includes three core fields: timestamp, horizontal displacement, and vertical displacement.

[0090] The displacement time series data is wirelessly transmitted to achieve real-time visualization of the displacement data and data interaction with downstream systems. The specific execution process is as follows: A wireless transmission and data interaction model is constructed, comprising five core parts: a data encoding module, a wireless transmission module, a data decoding module, a visualization module, and a platform interaction module. The data encoding module processes the displacement time series data... To perform packet encoding, first... It is split into multiple data blocks, each containing Each displacement data point is denoted as a data block. , Number the data blocks. Add a header, timestamp, checksum, and trailer to each data block to construct a data packet. The structure of the data packet is ,in For data packet header identifiers, This is the start timestamp of the data block. This is a cyclic redundancy check (CRC) code. The wireless transmission module, based on a preset wireless communication protocol, transmits data packets... Data is transmitted via a wireless communication link using a star topology. The master node receives data packets from all slave nodes, aggregates the data, and then sends it to the terminal. During transmission, a low-jitter communication interface is used to achieve synchronous data transmission and reception. The terminal's data decoding module verifies the received data packets. If the verification passes, the data header and footer are removed, and the data block is parsed out. By concatenating all data blocks according to their numbers, the displacement time series data can be reconstructed. The visualization module is based on A displacement visualization model is constructed, using a two-dimensional line graph to display the displacement trend over time. Horizontal and vertical displacement lines are plotted separately, and the range of the coordinate axes is automatically adjusted based on the extreme values ​​of the displacement data. The formula is as follows:

[0091]

[0092]

[0093] in , This represents the coordinate axis margin. The platform interaction module establishes a connection with the downstream structural health assessment platform through a preset communication interface, and uses a data interaction protocol to transmit displacement time series data. The data packets sent to the platform include a protocol header, data length, displacement data, checksum, and protocol trailer, ensuring the integrity and reliability of data transmission and providing continuous and accurate time-series data support for bridge structural health assessment.

[0094] This implementation achieves image input consistency through multi-dimensional standardization processing, constructs a multi-node microsecond-level timing synchronization system based on clock drift compensation, hardware-triggered synchronization, and frame buffer frequency stabilization, achieves sub-pixel-level accurate displacement calculation by combining a dual-mode displacement calculation model with frequency domain phase correlation and constrained attitude decomposition, enables low-latency real-time data processing through a localized edge computing architecture, and ensures the integrity and timing of data transmission through a wireless transmission and interaction model. Overall, it realizes high-precision, real-time, and synchronized wireless monitoring of bridge structure displacement.

[0095] Example 2 describes in detail a bridge structure displacement synchronous monitoring system based on wireless intelligent vision, used to implement the aforementioned wireless intelligent vision-based bridge structure displacement synchronous monitoring method, such as... Figure 5As shown, it specifically includes:

[0096] The image acquisition module acquires image data of the bridge structure monitoring area and performs format standardization processing on the image data, including noise reduction, size normalization, and color space conversion.

[0097] The node synchronization module performs clock synchronization and frame acquisition timing synchronization processing on multiple monitoring nodes. It eliminates clock deviation between nodes through linear drift compensation and achieves microsecond-level synchronization of each monitoring node by combining hardware-triggered synchronization and frame buffering, thus ensuring the timing consistency of image data of each node.

[0098] The displacement calculation module uses a target-independent tracking mode or an improved marker tracking mode to perform displacement calculation on the standardized image data, and obtains the sub-pixel displacement data of the bridge structure through frequency domain phase correlation analysis or constrained attitude decomposition algorithm.

[0099] The local computing module performs localized visual computing on subpixel displacement data on a low-power processor. It selects real-time or near-real-time computing mode according to monitoring requirements and outputs time-series displacement data of the bridge structure after time-series processing of the subpixel displacement data.

[0100] The wireless transmission module performs packet encoding on displacement time series data, completes the inter-node transmission and terminal sending of data packets through a wireless communication protocol, and realizes synchronous transmission and reception of collected and processed data through a low-jitter communication interface to ensure the integrity and timing of transmitted data.

[0101] The data interaction module receives displacement time series data sent by the wireless transmission module, realizes real-time visualization of displacement data, and completes data interaction with the downstream structural health assessment platform.

[0102] Furthermore, the image acquisition module is equipped with dual sensors, a global shutter and a rolling shutter, which can flexibly switch according to the lighting and vibration scenarios of the bridge monitoring. The sensors support multi-resolution acquisition and exposure, and the gain parameters are adjustable. After the acquired raw images are processed by Bayer format de-mosaic and Gaussian filtering for noise reduction, size normalization and RGB to grayscale color space conversion are completed. The output standardized grayscale image has a uniform range of pixel values, providing consistent input for subsequent displacement calculation. The module is also equipped with a zoom lens and protective components to adapt to the acquisition needs of different monitoring distances and complex environments of bridges in the field.

[0103] Furthermore, the node synchronization module is the core of multi-node synchronous monitoring. This module has three built-in sub-units: clock synchronization, hardware triggering, and frame buffer frequency stabilization. The clock synchronization sub-unit obtains the timestamp through two synchronization operations. Corresponding clock deviation The compensation value at any time is calculated using a linear drift compensation algorithm. The local clock of each node is calibrated in real time; the hardware triggering subunit establishes an interrupt signal connection between the global shutter sensor and the wireless node through the GPIO pin to realize synchronous triggering of frame acquisition of multiple nodes; the frame buffer frequency stabilization subunit stabilizes the acquisition frame rate through SDRAM frame buffer technology, and optimizes the calculation efficiency by combining dynamic window adjustment, and finally realizes microsecond-level timing synchronization of each monitoring node to ensure the timing consistency of image data acquired by multiple nodes.

[0104] Furthermore, the displacement calculation module incorporates two switchable sub-units: target-independent tracking and improved marker tracking, adapting to different needs for rapid bridge inspection and long-term monitoring. The target-independent tracking sub-unit eliminates the need for physical markers. It defines the monitoring ROI in a standardized image, performs a two-dimensional FFT transformation on the reference and current frames, calculates the frequency domain phase correlation matrix, and then uses IFFT transformation and peak centroid interpolation to calculate the subpixel displacement. The improved marker tracking sub-unit identifies the coordinates of marker corner points in the image and solves the homography matrix using linear least squares. Based on the plane vibration characteristics of bridges, a constrained attitude decomposition algorithm is introduced, combined with the camera intrinsic parameter matrix. The homography matrix decomposition is completed to calculate the actual displacement of the bridge. At the same time, the sub-unit has a built-in quality factor evaluation mechanism that automatically identifies and removes outliers based on pixel contrast and corner detection confidence, ensuring the reliability of subpixel displacement data.

[0105] Furthermore, the local computing module is deployed on a low-power embedded processor, integrating four major functions: task scheduling, core algorithm computation, data timing processing, and output control. It can perform core visual computation operations such as FFT calculation, phase correlation analysis, homography matrix solving, and camera pose estimation locally, without relying on cloud or remote computing resources. The module supports both real-time and near-real-time computing modes. In real-time mode, data is output immediately after single-frame displacement calculation. In near-real-time mode, frame buffering technology is used to improve the output frame rate, while resampling, sliding window filtering, and detrending processing are performed on continuous sub-pixel displacement data to output continuous and stable bridge structure displacement time series data. During the calculation process, dynamic task scheduling and fixed-point arithmetic optimization are used to control processor power consumption while ensuring computational accuracy, adapting to the long-term monitoring needs of unattended field operations.

[0106] Furthermore, the wireless transmission module is built on a star-shaped network architecture based on the ZigBee wireless communication protocol, integrating three functional units: data encoding, wireless transceiver, and low-jitter communication. It performs packet encoding on the displacement time series data output by the local computing module, adding a timestamp, checksum, and identification information to each data packet. Stable wireless transmission within 2km is achieved through signal enhancement technology. The module adopts a UART low-jitter communication interface to achieve synchronous transmission and reception of image acquisition data and displacement calculation data, effectively avoiding timing misalignment caused by transmission delay. At the same time, a cyclic redundancy check mechanism is used to verify the transmitted data packets to ensure the integrity and accuracy of data transmission.

[0107] Furthermore, the data interaction module consists of three sub-units: data reception, real-time visualization, and platform interaction. First, it receives displacement time-series data sent by the wireless transmission module and completes data parsing and verification. The real-time visualization sub-unit displays the dynamic trends of the bridge structure's horizontal and vertical displacements in real time through two-dimensional line graphs, numerical panels, and other formats. The coordinate axis range can be automatically adjusted according to the extreme values ​​of the displacement data. The platform interaction sub-unit establishes a connection with the downstream structural health assessment platform through a standardized data interface, completing the interactive transmission of displacement time-series data according to a preset protocol. This provides continuous and accurate data source support for the bridge structure's safety assessment, damage identification, and life prediction, achieving efficient utilization and closed-loop management of monitoring data.

[0108] This embodiment describes in detail how the system achieves high-precision non-contact monitoring of bridge structural displacement through multi-node microsecond-level synchronization and dual-mode displacement calculation; localized edge computing ensures real-time data processing, and wireless transmission and interaction modules enhance deployment flexibility and data integrity, making it adaptable to long-term unattended operation and rapid inspection scenarios, providing reliable support for bridge health assessment.

[0109] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.

Claims

1. A method for synchronous monitoring of bridge structure displacement based on wireless intelligent vision, characterized in that, include: Image data of the bridge structure monitoring area is collected, and the image data is processed to standardize the format. Clock synchronization and frame acquisition timing synchronization are performed on multiple monitoring nodes to ensure the timing consistency of image data acquired by each node. The displacement of the standardized image data is calculated by using a target-independent tracking mode or an improved marker tracking mode to obtain sub-pixel displacement data of the bridge structure. The subpixel displacement data is processed using localized visual computation to output displacement time series data of the bridge structure. The displacement time series data is wirelessly transmitted to enable real-time visualization of the displacement data and data interaction with downstream systems.

2. The bridge structure displacement synchronous monitoring method based on wireless intelligent vision according to claim 1, characterized in that, The clock synchronization and frame acquisition timing synchronization processing for multiple monitoring nodes includes: Perform two clock synchronization operations on each monitoring node to obtain the synchronization timestamp and the corresponding clock deviation value, and eliminate the clock deviation between each monitoring node through a linear drift compensation algorithm. The global shutter sensor is connected to the interrupt signal of the wireless node via hardware triggering to achieve hardware synchronization of frame acquisition by each monitoring node. By using frame buffering technology to stabilize the image acquisition frame rate of each monitoring node, and by combining dynamic window adjustment to optimize frequency domain calculation efficiency, and by periodically clearing the cache and garbage collection, microsecond-level synchronization of each monitoring node is achieved.

3. The bridge structure displacement synchronous monitoring method based on wireless intelligent vision according to claim 2, characterized in that, The elimination of clock skew between monitoring nodes using a linear drift compensation algorithm includes: Let the timestamps for the two clock synchronizations be... The corresponding clock offset value is The linear interpolation method is used to obtain the value at any time. Clock skew compensation value ; According to the clock skew compensation value Real-time compensation is performed on the local clocks of each monitoring node to eliminate timing errors caused by clock drift. The formula is: , This refers to any time when clock skew compensation is performed.

4. The bridge structure displacement synchronous monitoring method based on wireless intelligent vision according to claim 1, characterized in that, The displacement calculation of the standardized image data using the target-independent tracking mode includes: The first frame image before the load is applied is selected as the reference frame, and the image acquired during the monitoring process is used as the current frame to determine the region of interest for monitoring the two frames. Two-dimensional fast Fourier transforms are performed on the regions of interest of the reference frame and the current frame respectively, and the phase correlation matrix is ​​calculated through frequency domain phase correlation analysis; The phase correlation matrix is ​​subjected to subpixel interpolation using the peak centroid method to calculate the subpixel displacement of the monitoring area.

5. The bridge structure displacement synchronous monitoring method based on wireless intelligent vision according to claim 4, characterized in that, The phase correlation matrix is ​​calculated through frequency domain phase correlation analysis, and the formula is as follows: ,in, The phase correlation matrix, The result of the two-dimensional Fast Fourier Transform of the region of interest in the reference frame. This is the conjugate result of the two-dimensional Fast Fourier Transform of the region of interest in the current frame. These are the frequency components in the horizontal and vertical directions, respectively.

6. The bridge structure displacement synchronous monitoring method based on wireless intelligent vision according to claim 1, characterized in that, The method of employing an improved marker tracking mode to perform displacement calculation on the standardized image data includes: Identify the coordinates of the marked corner points in the image and obtain the homography matrix by solving the linear least squares method; Based on the motion characteristics of bridge vibration, out-of-plane motion is ignored and a constrained attitude decomposition algorithm is introduced to decompose the homography matrix and simplify the scale factor calculation process. By combining the camera intrinsic parameter matrix, the actual displacement of the bridge structure is obtained by solving the column vector of the homography matrix, and a quality factor is assigned to each frame of image to automatically identify and remove outliers in the displacement calculation.

7. The bridge structure displacement synchronous monitoring method based on wireless intelligent vision according to claim 6, characterized in that, The actual displacement of the bridge structure is obtained by combining the camera intrinsic parameter matrix and solving the column vector of the homography matrix, using the following formula: ,in, The column vectors of the homography matrix. For the camera intrinsic parameter matrix, Let be the column vector of the camera pose rotation matrix. Let be the camera pose translation vector. is the scale factor.

8. The bridge structure displacement synchronous monitoring method based on wireless intelligent vision according to claim 1, characterized in that, The localized visual computation processing of the subpixel displacement data, outputting the displacement time series data of the bridge structure, includes: Visual computation operations such as fast Fourier transform, phase correlation analysis, and camera pose estimation are performed locally on a low-power processor. Based on the actual needs of bridge monitoring, a real-time calculation mode or a near-real-time calculation mode is selected to perform time-series processing on subpixel displacement data and output continuous displacement time series data. The near-real-time calculation mode improves the data output frame rate through frame buffering technology.

9. The bridge structure displacement synchronous monitoring method based on wireless intelligent vision according to claim 1, characterized in that, Wireless transmission of the displacement time series data includes: The displacement time series data is packetized and encoded, and the encoded data packets are transmitted between nodes and sent to the terminal via a wireless communication protocol. During wireless transmission, a low-jitter communication interface is used to achieve synchronous transmission and reception of image acquisition data and displacement calculation data, ensuring the integrity and timing of the transmitted data.

10. A bridge structure displacement synchronous monitoring system based on wireless intelligent vision, characterized in that, include: The image acquisition module acquires image data of the bridge structure monitoring area and performs format standardization processing on the image data, including noise reduction, size normalization, and color space conversion. The node synchronization module performs clock synchronization and frame acquisition timing synchronization processing on multiple monitoring nodes. It eliminates clock deviation between nodes through linear drift compensation and achieves microsecond-level synchronization of each monitoring node by combining hardware-triggered synchronization and frame buffering, thus ensuring the timing consistency of image data of each node. The displacement calculation module uses a target-independent tracking mode or an improved marker tracking mode to perform displacement calculation on the standardized image data, and obtains the sub-pixel displacement data of the bridge structure through frequency domain phase correlation analysis or constrained attitude decomposition algorithm. The local computing module performs localized visual computing on subpixel displacement data on a low-power processor. It selects real-time or near-real-time computing mode according to monitoring requirements and outputs time-series displacement data of the bridge structure after time-series processing of the subpixel displacement data. The wireless transmission module performs packet encoding on displacement time series data, completes the inter-node transmission and terminal sending of data packets through a wireless communication protocol, and realizes synchronous transmission and reception of collected and processed data through a low-jitter communication interface to ensure the integrity and timing of transmitted data. The data interaction module receives displacement time series data sent by the wireless transmission module, realizes real-time visualization of displacement data, and completes data interaction with the downstream structural health assessment platform.

Images