Bridge widening splice joint width dynamic monitoring and early warning method based on computer vision

Abstract

The application discloses a bridge widening splicing joint width dynamic monitoring and early warning method based on computer vision and relates to the technical field of bridge intelligent monitoring and computer vision. The method synchronously collects multi-view images of edges on both sides of a joint body under the condition of no target by a multi-vision device, calculates six-degree-of-freedom spatial poses by three-dimensional reconstruction, forms a dynamic change track by continuous time sequence tracking, further introduces a support vector machine to intelligently classify by using the kinematic nature difference between normal expansion and contraction and abnormal dislocation, separates a creep dislocation component from total deformation, and triggers a graded early warning when the cumulative value of the cross-view synthesis exceeds the limit. The application breaks through the limitation of traditional schemes in single direction macroscopic monitoring, realizes synchronous capture of small rotation and dislocation, automatic distinction between normal fluctuation and abnormal damage, and graded early warning of early risks, and provides reliable technical support for the whole life cycle safe operation of a bridge expansion joint.

Description

Technical Field

[0001] This invention relates to the field of bridge intelligent monitoring and computer vision technology, specifically a method for dynamic monitoring and early warning of bridge widening splice width based on computer vision. Background Technology

[0002] In bridge engineering, expansion joints are crucial structures for ensuring structural safety and driving comfort. Their operational status directly affects the service life of the entire bridge. Abnormal deformation or damage to expansion joints often leads to serious structural damage and safety accidents. Therefore, continuous monitoring of expansion joints is of paramount importance. Currently, the widely used monitoring methods in engineering practice are: first, installing displacement sensors or attaching targets to the joint surface to measure the opening and closing degree of the joint. This method requires pre-positioning reference objects on the structure, which is often difficult to apply on a large scale to existing bridges, and the installed equipment can interfere with normal vehicle traffic; second, periodically conducting point cloud scans and comparing them with bridge modeling data. This method has a long measurement cycle, cumbersome data processing, and difficulty in capturing rapid changes in the joint. Most of these solutions can only observe the macroscopic movement of the joint in a single direction, making it difficult to simultaneously grasp the spatial posture and continuous evolution of the joint's two ends from multiple directions.

[0003] The sides of an expansion joint are not flat rigid bodies. Their edges undergo minute rotations and misalignments due to temperature fluctuations and repeated vehicle traffic. These changes in spatial orientation are difficult to detect in a two-dimensional planar view, yet they are crucial for determining whether structural damage has occurred to the joint. Due to the lack of effective means to simultaneously acquire the spatial orientation of the edges from multiple observation angles and continuously track its changes, engineers cannot distinguish whether fluctuations in the joint width are due to normal temperature expansion and contraction or irreversible misalignment at the edges. As these minute misalignments accumulate during continuous monitoring, they may eventually evolve into localized fractures at the joint-road interface, which existing monitoring methods cannot detect in their early stages.

[0004] Therefore, how to use multi-view stereo vision to simultaneously collect spatial attitude information of the two sides of the joint from different perspectives without relying on external targets, and continuously track its changing trends, so as to accurately distinguish between normal temperature fluctuations and abnormal creep displacement, has become a key issue in ensuring the long-term safe operation of bridges. Summary of the Invention

[0005] The purpose of this invention is to provide a computer vision-based method for dynamic monitoring and early warning of the width of bridge expansion joints, which enables dynamic monitoring and early warning of the width of expansion joints and the spatial posture of the two side edges without relying on external targets or interrupting traffic.

[0006] The objective of this invention can be achieved through the following technical solutions: This application provides a computer vision-based method for dynamic monitoring and early warning of bridge widening splice width, including the following steps: S1. Using multi-view vision devices deployed around the expansion joint, take multi-angle synchronous photos of the two sides of the joint without any external targets or physical references, and obtain a first set of images including the initial spatial state of the edges. S2. Based on the first image set, a stereo matching algorithm is used to perform sub-pixel level disparity calculation and depth information fusion on multi-view images to reconstruct a three-dimensional edge point cloud containing micron-level edge deformation information; then, a geometric topological model of the edges on both sides of the slit is constructed to calculate the initial spatial pose reflecting the small rotation and displacement of the edges in the initial state, and the two-dimensional image information is converted into three-dimensional spatial pose data that can be quantitatively analyzed. S3. Based on the initial spatial pose of multiple frames obtained from continuous time series, the longitudinal comparison and trajectory fitting algorithm is used to track the continuous change trajectory of edge micro-motion in three-dimensional space, and the discrete static spatial pose data is transformed into a complete dynamic change trajectory. S4. Based on the dynamic change trajectory, a support vector machine is introduced to classify and distinguish it in the multi-dimensional feature space. By utilizing the essential difference in motion between normal temperature expansion and contraction, which is elastic and reversible reciprocating motion, and abnormal creep displacement, which is irreversible unidirectional drift, the creep displacement component belonging to the abnormal state is accurately separated from the total deformation. S5. Based on the creep displacement component, extract its spatial displacement at multiple observation angles, synthesize the cumulative displacement value across viewing angles, and compare it with the preset structural safety threshold in real time; when the cumulative displacement value exceeds the limit, it is determined that the expansion joint has undergone irreversible abnormal displacement, and a graded structural damage early warning information for the risk of local fracture at the joint and road surface is generated, forming a complete intelligent monitoring closed loop from data acquisition to early warning output.

[0007] The beneficial effects of this invention are as follows: This invention utilizes multi-view vision devices deployed around expansion joints to perform sub-pixel-level parallax calculation and depth information fusion on multi-view images without any external targets or physical references. It employs stereo matching algorithms and iterative nearest-point algorithms, eliminating the need for drilling holes in the bridge deck to install sensors or attach targets, thus completely avoiding interference with vehicle traffic. This invention is suitable for both newly constructed bridges and existing bridges where traffic cannot be interrupted. Simultaneously, by spatially registering the three-dimensional edge point clouds of both sides of the joint, the invention decomposes the rotation angles around the three coordinate axes and the displacement along the three coordinate axes from the rigid body transformation matrix, forming a complete six-degree-of-freedom spatial pose vector. This overcomes the limitation of traditional solutions that can only observe macroscopic movement in a single direction, achieving synchronous and comprehensive capture of minute rotations and displacements at both ends of the joint. This invention continuously acquires initial spatial attitude data at a fixed sampling frequency. It uses a sliding window mechanism to segment and fit time-series data using polynomial fitting. The local motion trajectories of adjacent windows are then weighted and averaged over overlapping areas to form a continuous, dynamic trajectory without jumps. This mechanism ensures that the transient impact response and long-term, slow drift driven by temperature changes generated when each vehicle passes through the seam are fully recorded. It overcomes the problems of long measurement cycles, discrete data, and difficulty in capturing rapid changes and continuous evolution inherent in point cloud scanning schemes. This enables uninterrupted tracking of the entire process of spatial attitude changes at the seam edge, providing a complete temporal data foundation for distinguishing deformation properties. This invention introduces a support vector machine (SVM) to classify dynamic trajectories using multidimensional features. By extracting features such as displacement, velocity, acceleration, standard deviation of motion direction, rate of curvature change, and cumulative growth rate of the edge in three-dimensional space, and utilizing the essential difference in motion between normal temperature expansion and contraction (which exhibits elastic reversible reciprocating motion) and abnormal creep displacement (which exhibits irreversible unidirectional drift), an optimal classification hyperplane is constructed in a high-dimensional feature space. This accurately separates the creep displacement component from the total deformation, enabling automatic differentiation between normal temperature fluctuations and abnormal structural displacement. When the cumulative value of the creep displacement component after cross-view synthesis exceeds a preset grading threshold, the system automatically generates a corresponding level of early warning information. This allows engineers to obtain clear risk warnings before local fractures occur at the joint between the joint and the road surface, solving the problem that existing methods cannot detect the gradual accumulation of minute displacements leading to structural damage in the early stages. Attached Figure Description

[0008] To better understand and implement this application, the technical solution is described in detail below with reference to the accompanying drawings.

[0009] Figure 1 This is a flowchart illustrating the computer vision-based dynamic monitoring and early warning method for widening bridge splice joints, as provided in Embodiment 1 of this application. Detailed Implementation

[0010] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, exemplary embodiments will be described in detail below, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0011] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used herein are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0012] The following detailed description of the specific implementation methods, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided in detail.

[0013] Example 1, please refer to Figure 1 This embodiment takes a modular expansion joint on an elevated expressway in a certain city as the application object. The expansion joint has a span of L=8cm and is located at the junction of the main span and the approach bridge of a two-way four-lane bridge. The daily traffic volume on the bridge deck is about 50,000 vehicles. The area is located in a hot summer and cold winter region with an annual temperature difference of about 50℃. Specifically, it provides a method for dynamic monitoring and early warning of the width of bridge widening splice joints based on computer vision, including the following steps: S1. Using multi-view vision devices deployed around the expansion joint, without any external targets or physical references, take multi-angle synchronous photos of the two sides of the joint to obtain the first set of images containing the initial spatial state of the edges, which serves as the original data base for subsequent three-dimensional spatial reconstruction. Furthermore, step S1 specifically includes: S11. Based on the on-site span and lighting conditions of the expansion joint, at least three industrial cameras are installed around the expansion joint. The optical axes of each camera converge in the monitoring area where the two sides of the joint are located, forming a multi-view stereo vision observation network. The monitoring area is a strip-shaped range of 20cm before and after the joint and the road surface. The multi-view vision device includes at least three high-resolution industrial area array cameras. Each camera is arranged in a fan shape on a stable support above or to the side of the expansion joint. The angle between the optical axes of adjacent cameras is 15° to 45°. The imaging field of view of each camera together covers the full cross-sectional area of ​​the two sides of the expansion joint.

[0014] S12. Perform internal and external parameter calibration on each industrial camera, obtain the distortion coefficient, focal length, and rotation matrix and translation vector of each camera relative to the global world coordinate system, and optimize the reprojection error to less than 0.1 pixels through the bundled adjustment algorithm, and establish a unified three-dimensional space benchmark. In step S12: Zhang Zhengyou's checkerboard calibration method is used to calibrate the internal and external parameters of each industrial camera. The distortion coefficient, focal length, rotation matrix and translation vector of each camera relative to the global world coordinate system are calculated by corner detection and optimization algorithm. The parameters of each camera in the global world coordinate system are jointly optimized by the bundled adjustment algorithm.

[0015] S13. Connect each camera to the synchronization controller via a hardware trigger signal line. The synchronization controller sends trigger pulses to each camera simultaneously, so that each camera can synchronously acquire images under the same time reference. The image acquisition time synchronization error is less than 1 millisecond. Acquire synchronous image data of the two sides of the slit from different perspectives to form the first image set.

[0016] In step S13: during vehicle traffic gaps or continuous monitoring, the synchronization controller simultaneously sends trigger pulses to all cameras at a preset frequency of 20Hz. The image acquisition time synchronization error of each camera is less than 1 millisecond, ensuring that the multi-view images reflect the spatial state of the seam at the same moment.

[0017] Specifically, in this embodiment, the concrete edges of the expansion joint develop unique aggregate textures due to long-term vehicle traffic, which form natural "visual fingerprints" in the camera's field of view. During S11, engineers, based on the actual 8cm span of the expansion joint, arranged three 20-megapixel industrial area array cameras in a fan shape on a dedicated bracket approximately 1.5m above the joint. The optical axis angle between adjacent cameras was set to 25°, and the combined field of view of each camera covered a strip of approximately 40cm along both sides of the joint (20cm before and after the joint's junction with the road surface), ensuring that even when vehicles are traveling in adjacent lanes, they do not obstruct the view of any camera. During the execution of S12, after the initial installation, engineers placed a checkerboard calibration plate on the bridge deck for calibration. The calibration plate was placed flat on the surface of the expansion joint, and 20 sets of images were taken at different angles and heights. The focal length and distortion coefficient of each camera were calculated. In response to the slight vibrations generated by the bracket when heavy trucks pass by, the binding adjustment algorithm optimized the global reprojection error to 0.08 pixels, ensuring that the spatial distance error corresponding to the single pixel imaging error in the 20-megapixel image is controlled within 0.04mm. When executing S13, the synchronization controller triggers the three cameras to acquire data synchronously at a frequency of 20Hz. The actual measured synchronization error is 0.3ms. Even when the vehicle passes by at a speed of 80km / h, the vehicle only moves about 6.7mm within 0.3ms, and the resulting image blur is within an acceptable range, ensuring that the multi-view images accurately reflect the three-dimensional spatial state of the seam at the same moment.

[0018] S2. Based on the first image set, a stereo matching algorithm is used to perform sub-pixel level disparity calculation and depth information fusion on multi-view images to reconstruct a three-dimensional edge point cloud containing micron-level edge deformation information; then, a geometric topological model of the edges on both sides of the slit is constructed to calculate the initial spatial pose reflecting the small rotation and displacement of the edges in the initial state, and the two-dimensional image information is converted into three-dimensional spatial pose data that can be quantitatively analyzed. Furthermore, step S2 specifically includes: S21. Perform distortion correction and epipolar correction preprocessing on each view image in the first image set to eliminate imaging errors caused by lens distortion, so that the multi-view images meet the epipolar constraint conditions, and reduce the matching search of corresponding points in multiple views from two dimensions to one dimension. The distortion correction employs a correction algorithm based on radial and tangential distortion models, which inversely maps the spatial coordinates of each pixel in the image according to the distortion coefficients of each camera to generate a distortion-free standard image. The epipolar correction employs an epipolar constraint algorithm based on the fundamental matrix to transform the multi-view images to a common epipolar plane.

[0019] S22. Based on the corrected multi-view images, a semi-global stereo matching algorithm is used to dynamically program and solve the disparity along multiple directions. The left-right consistency test and uniqueness test are used to filter and remove mismatched points from the disparity calculation results, and a disparity map with sub-pixel accuracy is output. The semi-global stereo matching algorithm performs dynamic programming to solve disparity along 8 or 16 directions, introduces a smoothness constraint penalty term to deal with matching ambiguity in weak texture regions, and uses left-right consistency test and uniqueness test to screen and remove mismatched points from the disparity calculation results.

[0020] S23. The two-dimensional pixel coordinates under each viewpoint are fused with the corresponding depth information. Based on the principle of triangulation, the three-dimensional spatial coordinates of each pixel in the edge region of the seam are calculated, and the three-dimensional edge point cloud of the two sides of the seam is reconstructed. The spatial resolution of the three-dimensional edge point cloud reaches the micrometer level, and the point cloud density is not less than 100 points per square millimeter. In step S23: the edge region of the seam in the corrected multi-view image is pre-extracted using an edge detection operator. The two-dimensional coordinates and disparity values ​​of the corresponding pixels in the extracted region are combined with the camera projection matrix to calculate the X, Y, and Z coordinates of each pixel in three-dimensional space. The three-dimensional coordinate point sets corresponding to all effective pixels are stitched together to form the complete three-dimensional edge point cloud. The density of the three-dimensional edge point cloud is not less than 100 points per square millimeter.

[0021] S24. Based on the spatial distribution of the three-dimensional edge point cloud, the iterative nearest point algorithm is used to take the point cloud on one side of the seam as the reference point cloud and the point cloud on the other side as the target point cloud. The rigid body transformation matrix that minimizes the sum of squared distances between the target point cloud and the reference point cloud is solved iteratively. The relative rotation angle and relative displacement vector of the two sides of the seam are decomposed from the rigid body transformation matrix and used as the initial spatial attitude.

[0022] The iterative nearest point algorithm uses the point cloud on one side of the seam as the reference point cloud and the point cloud on the other side as the target point cloud. It iteratively solves for the rigid body transformation matrix that minimizes the sum of squared distances between the target point cloud and the reference point cloud. The rigid body transformation matrix contains a 3×3 rotation matrix and a 3×1 translation vector. The yaw angle, pitch angle, and roll angle around the X-axis, Y-axis, and Z-axis are decomposed from the rotation matrix as relative rotation angles. The relative displacement along the X-axis, Y-axis, and Z-axis is extracted from the translation vector, which together constitute a complete spatial pose vector with six degrees of freedom.

[0023] Specifically, after obtaining the synchronized image, the system immediately performs 3D reconstruction. When executing S21, the original image is deformed into a barrel shape at the edge of the concrete aggregate due to lens distortion. For example, a straight steel bar at the edge of the joint appears slightly curved in the image. The distortion correction algorithm performs inverse mapping of the spatial coordinates of each pixel based on the calibrated radial distortion coefficients k1, k2 and tangential distortion coefficients p1, p2 to restore the true straight line shape. Epipolar correction further transforms the three-view images to a common epipolar plane. The pixel positions of the same point on the surface of the concrete aggregate may differ by hundreds of rows in the three images. After correction, they are all aligned to the same row. The matching search is reduced from a two-dimensional plane to a one-dimensional row search, and the computation is reduced from millions of times to hundreds of times. In execution S22, taking into account the rich and rough texture of the concrete surface, a semi-global stereo matching algorithm performs dynamic programming along eight directions, calculating the sub-pixel disparity value for each pixel at the joint edge. For areas where reflections may cause sparse texture at the edge of the steel expansion joint, the algorithm eliminates mismatches through left-right consistency checks. For example, if a pixel in the left image matches the right image, and the position difference exceeds 2 pixels when matched back in the reverse direction, it is considered a mismatch and is eliminated. The final output disparity map achieves a disparity accuracy of 0.1 pixels at the joint edge, corresponding to a spatial depth accuracy of approximately 0.02 mm. In execution S23, the triangulation principle converts the disparity map into three-dimensional coordinates. Using one viewpoint from three cameras as the main view, the X, Y, and Z coordinates of each pixel at the edge in the world coordinate system are calculated using the projection matrices of each camera. Within a 40cm × 30cm field of view, the point cloud density reaches 120 points per square millimeter, which means that within 1cm of the joint edge... 2 It contains approximately 12,000 three-dimensional coordinate points, and the sand-grain-level texture of the concrete surface is accurately reconstructed; When executing S24, the iterative nearest point algorithm performs rigid body registration between the left edge point cloud and the right edge point cloud. During the first registration, a protruding aggregate in the left edge point cloud is temporarily matched with a certain area in the right edge point cloud. After multiple iterations and convergence, the relative rotation of the two edges is decomposed from the final rigid body transformation matrix as 0.02° around the X-axis (corresponding to a slight warping of the concrete block), and the displacement is 0.12mm in the bridge axis, 0.03mm in the transverse direction, and 0.05mm in the vertical direction, forming a complete initial spatial attitude with six degrees of freedom.

[0024] S3. Based on the initial spatial pose of multiple frames obtained from continuous time series, the longitudinal comparison and trajectory fitting algorithm are used to track the continuous change trajectory of edge micro-motion in three-dimensional space, and the discrete static spatial pose data is transformed into a complete dynamic change trajectory, providing time series samples for subsequent feature classification. Furthermore, step S3 specifically includes: S31. The initial spatial attitude is continuously acquired at a fixed sampling frequency of 10Hz to 50Hz to form a time series dataset consisting of timestamps and corresponding three-dimensional spatial pose vectors. The three-dimensional spatial pose vectors include rotation angles around three coordinate axes and displacement along three coordinate axes. The fixed sampling frequency is 30Hz, and each frame of data in the time series dataset contains a timestamp and a corresponding six-dimensional spatial pose vector (Rx, Ry, Rz, Tx, Ty, Tz), where Rx, Ry, and Rz are the rotation angles around the bridge axis, transverse direction, and vertical direction, respectively, and Tx, Ty, and Tz are the displacement amounts along the bridge axis, transverse direction, and vertical direction, respectively.

[0025] S32. The time series dataset is segmented and truncated using a sliding window mechanism. The window width is 50 to 200 frames, the sliding step size is 10 to 50 frames, and there is an overlapping area between adjacent windows. The sliding window has a width of 50 frames, a sliding step of 10 frames, and an overlap of 40 frames between adjacent windows to ensure the continuity and smoothness of the trajectory fitting. The window width corresponds to a monitoring duration of 1.67 seconds.

[0026] S33. Perform trajectory fitting on the spatial pose data within each window. Use third- to fifth-order polynomials to perform least-squares fitting on the spatial pose data of each frame within the window. The root mean square error of the fitting result is used as the confidence evaluation index of the motion trajectory within the window. The third to fifth order polynomial fitting is a least-squares approximation of the spatial pose data of each frame within the window according to the time series. When the root mean square error of the fitting result exceeds a preset threshold, the window is marked as a low-confidence window, prompting subsequent steps to reduce the weight of the data in the window during classification.

[0027] S34. The local motion trajectories of adjacent windows are spliced ​​together in time. A fusion method based on weighted average of overlapping regions is used to smooth the attitude data of the fitted trajectories of adjacent windows in the overlapping region, forming a continuous dynamic change trajectory without jumps.

[0028] Specifically, the fusion method based on weighted average of overlapping regions involves assigning weights to adjacent windows within their overlapping regions according to their time distance from the center of each window, with higher weights closer to the center. The fitted pose values ​​of the two windows at the same time are then weighted and averaged to obtain the final pose value at that time, forming a continuous dynamic change trajectory without jumps.

[0029] Specifically, during S31 execution, the system continuously collects spatial attitude data at a frequency of 30Hz. In real bridge operation scenarios, the temperature fluctuates periodically throughout the day. At 5:00 AM, the temperature is lowest (approximately 5°C), and the joint opens to its maximum (bridge axial displacement is approximately -2.5mm, representing an increase in joint width); at 2:00 PM, the temperature is highest (approximately 38°C), and the joint closes to its minimum (bridge axial displacement is approximately +1.2mm). Each frame of six-dimensional attitude data (Rx, Ry, Rz, Tx, Ty, Tz) is accompanied by a millisecond-level timestamp and stored in the database. When executing S32, the sliding window width is set to 90 frames (corresponding to 3 seconds), the sliding step size is 15 frames (corresponding to 0.5 seconds), and adjacent windows overlap by 75 frames. The reason for choosing a 3-second window is that the impact time of a truck traveling at 60 km / h on the seam is about 0.3 to 0.5 seconds, and a 3-second window is sufficient to completely contain the impact response and its subsequent vibration decay process; at the same time, the temperature change within 3 seconds is extremely small (<0.01℃), and the temperature expansion and contraction can be regarded as a constant bias within this window, which does not affect the accuracy of trajectory fitting. When executing S33, a fourth-order polynomial is used to fit the bridge axial displacement Tz in each window. In the actual test scenario, when a vehicle passes through the gap, it generates a rapid downward displacement pulse and then rebounds. This pulse is superimposed on the slow changes in temperature expansion and contraction. The fourth-order polynomial can accurately fit the rising and falling edges of the pulse, and the root mean square error of the fit is usually less than 0.02 mm. If a window experiences a data jump due to instantaneous occlusion, the fitting error will suddenly increase to more than 0.1 mm. This window is marked as low confidence and triggers a reduction in weight. When executing S34, adjacent windows are weighted according to their time distance from the center within the 75-frame overlapping area. For example, in the 5th frame within the overlapping area, which is closer to the center of the left window, the fitted value of the left window is weighted at 0.8 and the right window at 0.2. After weighted averaging, a smooth transition of continuous values ​​is obtained, ultimately forming a complete dynamic change trajectory spanning several hours or even days. This clearly shows the dynamic response of each vehicle as it passes through the seam and the slow drift driven by temperature.

[0030] S4. Based on the dynamic change trajectory, a support vector machine is introduced to classify and distinguish it in the multi-dimensional feature space. By utilizing the essential difference in motion between normal temperature expansion and contraction, which is elastic and reversible reciprocating motion, and abnormal creep displacement, which is irreversible unidirectional drift, the creep displacement component belonging to the abnormal state is accurately separated from the total deformation. Furthermore, step S4 specifically includes: S41. Extract the displacement, displacement velocity, displacement acceleration, motion direction persistence, motion trajectory curvature change rate, motion direction standard deviation in three-dimensional space, and cumulative growth rate of deformation per unit time from the dynamic change trajectory as multi-dimensional feature vectors. The multidimensional feature vector further includes the curvature change rate of the edge motion trajectory, the standard deviation of the motion direction in three-dimensional space, and the cumulative growth rate of the deformation per unit time. The curvature change rate reflects the degree of abrupt change in the edge motion direction, the standard deviation reflects the degree of dispersion of the motion, and the cumulative growth rate reflects the strength of the unidirectional accumulation trend of deformation.

[0031] S42. Using the temperature expansion and contraction samples and creep displacement samples already labeled in the historical monitoring data as the training set, the training set includes historical monitoring data under different seasons, different temperature ranges and different load conditions, the support vector machine is trained to construct a classification hyperplane for normal fluctuations and abnormal displacements. The training set includes data from different seasons, different temperature ranges, and historical monitoring data under different load conditions. The temperature range covers -10℃ to 40℃, and all training samples have been manually verified and labeled.

[0032] S43. Input the multidimensional feature vector of the current monitoring period into the trained support vector machine for classification and discrimination. Use the radial basis kernel function to map the multidimensional feature vector to a high-dimensional feature space. Construct the optimal classification hyperplane in the high-dimensional feature space. Determine whether the current deformation belongs to normal temperature expansion and contraction or abnormal creep displacement based on the position of the feature vector relative to the classification hyperplane. Output the distance from the feature vector to the classification hyperplane as the classification confidence. The classification confidence is the distance from the feature vector to the optimal classification hyperplane. When the classification confidence is lower than a preset threshold, a manual review mechanism is triggered. Instead of directly outputting the classification result, the system is switched to a pending review state. After manual confirmation, the training set is updated and the support vector machine is retrained.

[0033] S44. When the creep fault is determined to be abnormal and the classification confidence exceeds the preset threshold, the estimated value of normal temperature expansion and contraction during the same period is subtracted from the total deformation to extract the creep fault component, and its start time and spatial orientation are recorded.

[0034] The estimated value of the normal temperature expansion and contraction during the same period is obtained by calculating through a pre-established temperature-deformation regression model. The temperature-deformation regression model is established by fitting the synchronously collected ambient temperature data and the normal temperature expansion and contraction samples in the historical monitoring data. It is used to separate the elastic deformation component caused by temperature from the current total deformation, thereby extracting the irreversible creep displacement component.

[0035] Specifically, when executing S41, the system extracts feature vectors with more than 7 dimensions from the dynamic trajectory. In the scenario, monitoring data over a period of 3 hours shows that the joint slowly closes with the temperature during the afternoon high-temperature period (the axial displacement of the bridge decreases linearly at a rate of 0.005 mm / min). At the same time, due to the uneven distribution of sunlight on the bridge surface, it experiences an axial rotation of approximately 0.01°. These features are characterized by a low rate of curvature change (<0.001°). 2 The small standard deviation (dispersion of motion direction along the bridge axis <5°) indicates a typical temperature-induced expansion and contraction pattern. However, after three consecutive heavy-duty trucks (total weight exceeding 80 tons) passed, a residual displacement of 0.06 mm appeared on the right edge and did not recover, accompanied by a sudden increase in the standard deviation of the motion direction to 22° (a significant transverse component appeared), and the rate of curvature change increased to 0.05°. 2 / s, these features constitute the typical fingerprint of abnormal creep; When executing S42, the system trains a support vector machine using monitoring data collected over the past year. The training set includes 1500 samples each from winter (-5℃ to 5℃) and summer (30℃ to 40℃), as well as 800 artificially simulated creep fault samples. All samples are labeled with a labeling accuracy of no less than 95%. When executing S43, the radial basis function is used to map multidimensional features to a high-dimensional space. The constructed classification hyperplane can separate normal expansion and contraction from abnormal faults with an accuracy of 99.2%. When the feature vector at the current moment is input, its distance to the classification hyperplane is 0.68 (exceeding the preset threshold of 0.5), and it is judged as an abnormal creep fault. When executing S44, the system calls the temperature-deformation regression model based on the temperature data of the same period (current temperature 35℃) to calculate the estimated displacement caused by thermal expansion as 0.15mm. After subtracting from the total deformation of 0.23mm, the system extracts the creep displacement component of 0.08mm and records its start time as 13:47 on the same day, with its spatial orientation being the right edge near the shoulder.

[0036] S5. Based on the creep displacement component, extract its spatial displacement at multiple observation angles, synthesize the cumulative displacement value across viewing angles, and compare it with the preset structural safety threshold in real time; once the cumulative displacement value exceeds the limit, it is determined that the expansion joint has undergone irreversible abnormal displacement, and then a graded structural damage early warning information for the risk of local fracture at the joint and road surface is generated, forming a complete intelligent monitoring closed loop from data acquisition to early warning output.

[0037] Furthermore, step S5 specifically includes: S51. Extract the spatial displacement observed from each camera viewpoint from the creep displacement component, and use multi-view geometric constraints to verify the consistency of the displacement from each viewpoint. If the displacement observed from a certain viewpoint deviates from other viewpoints by more than a set tolerance threshold, it will be removed. The tolerance threshold is set to 0.02 mm. When the deviation of the spatial displacement observed from a certain perspective from the observations of other perspectives exceeds 0.02 mm, the observation from that perspective is determined to be affected by local occlusion or environmental interference and is discarded. When the deviation between the observations from each perspective is less than 0.02 mm, all of them pass the consistency check and enter the subsequent vector synthesis.

[0038] S52. The verified multi-view spatial displacement is uniformly converted to the world coordinate system for vector synthesis. The axial, transverse and vertical displacement components of the bridge are accumulated respectively to obtain the accumulated displacement value in each direction. Then, the total accumulated displacement value in three-dimensional space is synthesized. The step of accumulating the axial, transverse, and vertical displacement components of the bridge is to uniformly transform the verified multi-view spatial displacement to a world coordinate system with the center point of the joint as the origin, and then accumulate the creep displacement in each direction frame by frame to obtain the accumulated displacement values ​​Tx_accumulated, Ty_accumulated, and Tz_accumulated in each direction. These are then combined to form the total accumulated displacement value D_total in three-dimensional space: D_total = √(Tx_accumulated). 2 +Ty_cumulative 2 +Tz_Accumulated 2 ).

[0039] S53. The cumulative value of the misalignment is compared with the preset structural safety threshold in real time. The structural safety threshold is set according to the design allowable deformation of the expansion joint and the bridge structural health assessment standard. The first preset threshold is 40% of the design allowable value, and the second preset threshold is 70% of the design allowable value. Wherein, the allowable deformation amount is the maximum allowable displacement amount specified in the expansion joint design document, the first preset threshold is 40% of the allowable design value, corresponding to the warning trigger line of the attention level; the second preset threshold is 70% of the allowable design value, corresponding to the warning trigger line of the alert level; when the cumulative displacement value exceeds 100% of the allowable design value, the danger level warning is automatically triggered.

[0040] S54. When the cumulative value of the fault exceeds the first preset threshold, a level 1 warning message is generated; when it exceeds the second preset threshold, a level 2 warning message is generated; when the cumulative value of the fault rapidly crosses the two thresholds within a preset time period, it is automatically upgraded to a level 3 warning message. Each level of warning message corresponds to a different structural damage risk level and emergency response measures. The warning message includes the abnormal fault start time, spatial location, and classification confidence level.

[0041] The tiered early warning information also includes the historical trend curve of the cumulative fault value and the predicted fault speed; the first-level early warning information is pushed to bridge maintenance engineers through monitoring platform pop-ups and SMS messages; the second-level early warning information adds telephone voice notification on the basis of the former; the third-level early warning information activates the emergency response process, including local traffic control suggestions and on-site inspection instructions from professional testing teams.

[0042] The edges of the expansion joint undergo micron-level rotation and displacement under the repeated action of temperature rise and fall and vehicle load. The initial spatial posture, dynamic change trajectory and creep displacement components are all acquired without external targets and physical references. The multi-view vision device continuously outputs six-degree-of-freedom pose data during continuous monitoring, forming a complete intelligent monitoring closed loop from data acquisition, three-dimensional reconstruction, time-series tracking, feature classification to hierarchical early warning.

[0043] Specifically, when executing S51, the system calculates the creep displacement of the right edge relative to the left edge from three camera perspectives. The observed values ​​are 0.082mm from the left perspective, 0.079mm from the middle perspective, and 0.085mm from the right perspective. The deviations of the three values ​​are less than 0.006mm, which is far below the tolerance threshold of 0.02mm, and all of them pass the consistency check. When executing S52, the displacements verified from the three perspectives were converted to the world coordinate system and then vector-synthesized, resulting in an axial displacement of 0.07 mm, a transverse displacement of 0.03 mm, and a vertical displacement of 0.02 mm. The total cumulative displacement was √(0.07 mm). 2 +0.03 2 +0.02 2 The cumulative value of the fault was approximately 0.079 mm. In subsequent continuous monitoring, the cumulative value of the fault continued to increase at a rate of approximately 0.15 mm every 10 days. When executing S53, the system substitutes the design allowable misalignment of the expansion joint of 5mm, the first preset threshold is 2mm (40%), and the second preset threshold is 3.5mm (70%). After continuous monitoring, the cumulative misalignment value reaches 1.8mm on the 45th day, which is close to the first-level threshold but has not yet exceeded the limit; on the 52nd day, the cumulative misalignment value exceeds 2.0mm, triggering a first-level warning. When executing S54, the system automatically generates a Level 1 warning message and pushes it to the bridge maintenance engineer. The message includes: "The cumulative creep displacement of the right edge of the expansion joint (according to the calibrated location, such as KXX+YYY) has reached 2.05mm (exceeding the first threshold of 2.0mm). The abnormal displacement started at a certain time point (e.g., N hours and N minutes on X month X day, 202X), and the spatial orientation is the right edge near the shoulder. The classification confidence level is 0.82. It is recommended to increase the observation frequency and arrange routine inspections." If the displacement continues to accelerate and quickly reaches 3.5mm within the next two weeks, the system will automatically upgrade to an orange Level 2 warning. If the displacement growth rate exceeds 0.5mm within a week and a grade jump occurs simultaneously (directly skipping two levels), the system will automatically trigger a red Level 3 warning, directly notifying the bridge management unit to initiate an emergency response, including local traffic control and on-site investigation by a professional team, to achieve early identification and proactive intervention of the risk of local fracture at the joint and pavement connection.

[0044] Example 2 differs from Example 1 in that it further introduces temperature-displacement correlation modeling and an adaptive temperature compensation mechanism. By decoupling and compensating for temperature effects in the visual measurement results, it eliminates spurious deformation interference caused by changes in ambient temperature. The following only describes the steps that differ from Example 1; identical parts will not be repeated.

[0045] In step S1, the difference from Example 1 is as follows: While deploying the multi-view vision device, an ambient temperature sensor and a seam surface infrared temperature sensor are deployed on the same mounting plane as the camera bracket. The data acquisition frequency of the temperature sensor is synchronized with the camera sampling frequency to ensure that each frame of image corresponds to the seam and ambient temperature data at the same moment. While acquiring the first image set, the temperature dataset corresponding to the moment is acquired simultaneously. The temperature dataset includes the ambient air temperature and the seam surface temperature. Each frame of data is expanded to include the synchronously acquired temperature data on the basis of the original six-dimensional spatial pose vector.

[0046] In step S3, the difference from Example 1 is as follows: Based on time-series trajectory analysis, temperature-displacement correlation modeling is further introduced. Each window contains not only spatial attitude data but also a temperature data sequence within that window. The temperature data within the window is linearly fitted to obtain the average temperature and temperature change rate within that window. Based on long-term accumulated historical monitoring data, a temperature-displacement regression model between the axial displacement of the expansion joint bridge and the effective temperature is established. The effective temperature is obtained by weighted fusion of ambient air temperature and joint surface temperature. The weighting coefficients are determined by maximizing the correlation between temperature and displacement. The regression model uses a linear regression method to establish the response relationship of displacement with respect to the effective temperature. After the regression model is established, the model parameters are periodically updated using newly collected data to adapt to seasonal changes and the long-term evolution of structural performance.

[0047] In step S4, the difference from Example 1 is as follows: A temperature compensation mechanism is introduced, extracting temperature data corresponding to each window as auxiliary features. When abnormal creep displacement is determined and the classification confidence exceeds a preset threshold, the estimated value of normal temperature expansion and contraction during the same period is subtracted from the total deformation to extract the creep displacement component. The estimated value of normal temperature expansion and contraction during the same period adopts a dual calculation strategy: The first estimation is based on a temperature-displacement regression model, which calculates the theoretical displacement caused by temperature based on the effective temperature of the current window. The second estimation is based on the statistical mean of normal temperature expansion and contraction samples from the same historical period (same season, similar temperature range). The final temperature expansion and contraction estimate is the weighted average of the two estimates, with the weights dynamically adjusted according to the current classification confidence level. When the confidence level is high, the weight of the first estimation level is increased, and when the confidence level is low, the weight of the second estimation level is increased to enhance robustness.

[0048] After subtracting the estimated value of temperature expansion and contraction from the total deformation, the irreversible creep fault component is extracted, and its start time and spatial orientation are recorded.

[0049] In step S5, the difference from Example 1 is as follows: Before comparing the cumulative displacement value with the preset structural safety threshold in real time, the temperature compensation effect is first verified: the residual of the temperature-displacement regression model in the current window is calculated. If the residual exceeds the preset allowable range, it indicates that the current temperature-displacement relationship has changed significantly, which may indicate that the structural stiffness has changed or the regression model needs to be updated. The system automatically marks the data in this period as "needs to be reviewed" and triggers the model update process. If the residual is within the normal range, the subsequent threshold comparison process is entered.

[0050] In addition to the content of Example 1, the early warning information also includes temperature compensation-related parameters (current effective temperature, temperature-displacement regression model residuals, temperature compensation amount, etc.) to help engineers comprehensively assess the credibility and urgency of the early warning.

[0051] Technical effects of this embodiment: Based on Example 1, this embodiment further achieves the following technical effects by introducing temperature-displacement correlation modeling and an adaptive temperature compensation mechanism: First, it establishes a quantitative response relationship between the expansion joint displacement field and the ambient temperature field, enabling a more accurate distinction between normal elastic deformation caused by temperature and irreversible displacement caused by structural damage, effectively avoiding the misjudgment of pseudo-deformation caused by large seasonal temperature differences as abnormal displacement; Second, through temperature compensation residual analysis, it achieves continuous monitoring of the effectiveness of the temperature-displacement regression model itself, and can promptly detect problems such as changes in structural stiffness when the residual abnormally increases; Third, the dual temperature estimation strategy enhances the robustness of temperature compensation, especially maintaining high compensation accuracy under extreme weather conditions; Fourth, temperature compensation-related parameters are integrated into the early warning information, enabling engineers to more comprehensively assess the credibility of the early warning, improving the practicality and decision support capabilities of the monitoring system.

[0052] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any brief modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for dynamic monitoring and early warning of bridge widening splice width based on computer vision, characterized in that: Includes the following steps: S1. Using multi-view vision devices deployed around the expansion joint, take multi-angle synchronous photos of the two sides of the joint without any external targets or physical references, and obtain a first set of images including the initial spatial state of the edges. S2. Based on the first image set, a stereo matching algorithm is used to perform sub-pixel level disparity calculation and depth information fusion on multi-view images to reconstruct a three-dimensional edge point cloud containing micron-level edge deformation information; then, a geometric topological model of the edges on both sides of the slit is constructed to calculate the initial spatial pose reflecting the small rotation and displacement of the edges in the initial state, and the two-dimensional image information is converted into three-dimensional spatial pose data that can be quantitatively analyzed. S3. Based on the initial spatial pose of multiple frames obtained from continuous time series, the longitudinal comparison and trajectory fitting algorithm is used to track the continuous change trajectory of edge micro-motion in three-dimensional space, and the discrete static spatial pose data is transformed into a complete dynamic change trajectory. S4. Based on the dynamic change trajectory, a support vector machine is introduced to classify and distinguish it in the multi-dimensional feature space. By utilizing the essential difference in motion between normal temperature expansion and contraction, which is elastic and reversible reciprocating motion, and abnormal creep displacement, which is irreversible unidirectional drift, the creep displacement component belonging to the abnormal state is accurately separated from the total deformation. S5. Based on the creep displacement component, extract its spatial displacement at multiple observation angles, synthesize the cumulative displacement value across viewing angles, and compare it with the preset structural safety threshold in real time; when the cumulative displacement value exceeds the limit, it is determined that the expansion joint has undergone irreversible abnormal displacement, and a graded structural damage early warning information for the risk of local fracture at the joint and road surface is generated, forming a complete intelligent monitoring closed loop from data acquisition to early warning output.

2. The method for dynamic monitoring and early warning of bridge widening splice joint width based on computer vision according to claim 1, characterized in that: Step S1 specifically includes: S11. Based on the on-site span and lighting conditions of the expansion joint, at least three industrial cameras are installed around the expansion joint. The optical axes of each camera converge in the monitoring area where the two sides of the joint are located, forming a multi-view stereo vision observation network. The monitoring area is a strip-shaped range of 20cm before and after the joint and the road surface. S12. Perform internal and external parameter calibration on each industrial camera, obtain the distortion coefficient, focal length, and rotation matrix and translation vector of each camera relative to the global world coordinate system, and optimize the reprojection error to meet the high-precision calibration requirements through the bundled adjustment algorithm, and establish a unified three-dimensional space benchmark. S13. Connect each camera to the synchronization controller via a hardware trigger signal line. The synchronization controller sends trigger pulses to each camera simultaneously, so that each camera can synchronously acquire images under the same time reference, obtain synchronous image data of the two sides of the slit from different perspectives, and form the first image set.

3. The method for dynamic monitoring and early warning of bridge widening splice joint width based on computer vision according to claim 2, characterized in that: The multi-view vision device includes at least three high-resolution industrial area array cameras. Each camera is arranged in a fan shape on a stable support above or to the side of the expansion joint. The imaging field of each camera together covers the full cross-sectional area of ​​the two sides of the expansion joint. The angle between the optical axes of adjacent cameras is configured so that the images from each viewpoint have sufficient parallax information.

4. The method for dynamic monitoring and early warning of bridge widening splice joint width based on computer vision according to claim 2, characterized in that: In step S12: Zhang Zhengyou's checkerboard calibration method is used to calibrate the internal and external parameters of each industrial camera. The distortion coefficient, focal length, rotation matrix and translation vector of each camera relative to the global world coordinate system are calculated by corner detection and optimization algorithm. The parameters of each camera in the global world coordinate system are jointly optimized by the bundled adjustment algorithm.

5. The method for dynamic monitoring and early warning of bridge widening splice width based on computer vision according to claim 1, characterized in that: Step S2 specifically includes: S21. Perform distortion correction and epipolar correction preprocessing on each view image in the first image set to eliminate imaging errors caused by lens distortion, so that the multi-view images meet the epipolar constraint conditions, and reduce the matching search of corresponding points in multiple views from two dimensions to one dimension. S22. Based on the corrected multi-view images, a semi-global stereo matching algorithm is used to dynamically program and solve the disparity along multiple directions. A consistency check is used to filter and remove mismatched points from the disparity calculation results, and a disparity map with sub-pixel accuracy is output. S23. The two-dimensional pixel coordinates under each viewpoint are fused with the corresponding depth information. Based on the principle of triangulation, the three-dimensional spatial coordinates of each pixel in the edge region of the seam are calculated, and the three-dimensional edge point cloud of the two sides of the seam is reconstructed. S24. Based on the spatial distribution of the three-dimensional edge point cloud, the iterative nearest point algorithm is used to take the point cloud on one side of the seam as the reference point cloud and the point cloud on the other side as the target point cloud. The rigid body transformation matrix that minimizes the sum of squared distances between the target point cloud and the reference point cloud is solved iteratively. The relative rotation angle and relative displacement vector of the two sides of the seam are decomposed from the rigid body transformation matrix and used as the initial spatial attitude.

6. The method for dynamic monitoring and early warning of bridge widening splice joint width based on computer vision according to claim 1, characterized in that: Step S3 specifically includes: S31. The initial spatial attitude is continuously acquired at a fixed sampling frequency to form a time series dataset consisting of timestamps and corresponding three-dimensional spatial pose vectors. The three-dimensional spatial pose vectors include rotation angles around three coordinate axes and displacement along three coordinate axes. S32. The time series dataset is segmented and truncated using a sliding window mechanism, with overlapping areas between adjacent windows; S33. Perform trajectory fitting on the spatial pose data within each window. Use a polynomial to perform least squares fitting on the spatial pose data of each frame within the window. The root mean square error of the fitting result is used as the confidence evaluation index of the motion trajectory within the window. S34. The local motion trajectories of adjacent windows are spliced ​​together in time. A fusion method based on weighted average of overlapping regions is used to smooth the attitude data of the fitted trajectories of adjacent windows in the overlapping region, forming a continuous dynamic change trajectory without jumps.

7. The method for dynamic monitoring and early warning of bridge widening splice width based on computer vision according to claim 1, characterized in that: Step S4 specifically includes: S41. Extract the displacement, displacement velocity, displacement acceleration, motion direction persistence, motion trajectory curvature change rate, motion direction standard deviation in three-dimensional space, and cumulative growth rate of deformation per unit time from the dynamic change trajectory as multi-dimensional feature vectors. S42. Using the temperature expansion and contraction samples and creep displacement samples already labeled in the historical monitoring data as the training set, the training set includes historical monitoring data under different seasons, different temperature ranges and different load conditions, the support vector machine is trained to construct a classification hyperplane for normal fluctuations and abnormal displacements. S43. Input the multidimensional feature vector of the current monitoring period into the trained support vector machine for classification and discrimination. Use a kernel function to map the multidimensional feature vector to a high-dimensional feature space. Construct the optimal classification hyperplane in the high-dimensional feature space. Determine whether the current deformation belongs to normal temperature expansion and contraction or abnormal creep displacement based on the position of the feature vector relative to the classification hyperplane. Output the distance from the feature vector to the classification hyperplane as the classification confidence. S44. When the creep fault is determined to be abnormal and the classification confidence exceeds the preset threshold, the estimated value of normal temperature expansion and contraction during the same period is subtracted from the total deformation to extract the creep fault component, and its start time and spatial orientation are recorded.

8. The method for dynamic monitoring and early warning of bridge widening splice joint width based on computer vision according to claim 7, characterized in that: The classification confidence is the distance from the feature vector to the optimal classification hyperplane. When the classification confidence is lower than a preset threshold, a manual review mechanism is triggered. Instead of directly outputting the classification result, the system is switched to a pending review state. After manual confirmation, the training set is updated and the support vector machine is retrained.

9. The method for dynamic monitoring and early warning of bridge widening splice joint width based on computer vision according to claim 7, characterized in that: The multidimensional feature vector also includes the curvature change rate of the edge motion trajectory, the standard deviation of the motion direction in three-dimensional space, and the cumulative growth rate of the deformation per unit time. The curvature change rate reflects the degree of abrupt change in the edge motion direction, the standard deviation reflects the degree of dispersion of motion, and the cumulative growth rate reflects the strength of the trend of unidirectional accumulation of deformation.

10. The method for dynamic monitoring and early warning of bridge widening splice width based on computer vision according to claim 1, characterized in that: Step S5 specifically includes: S51. Extract the spatial displacement observed from each camera viewpoint from the creep displacement component, and use multi-view geometric constraints to verify the consistency of the displacement from each viewpoint. If the displacement observed from a certain viewpoint deviates from other viewpoints by more than a set tolerance threshold, it will be removed. S52. The verified multi-view spatial displacement is uniformly converted to the world coordinate system for vector synthesis. The axial, transverse and vertical displacement components of the bridge are accumulated respectively to obtain the accumulated displacement value in each direction. Then, the total accumulated displacement value in three-dimensional space is synthesized. S53. The cumulative value of the misalignment is compared with the preset structural safety threshold in real time. The structural safety threshold is set according to the design allowable deformation of the expansion joint and the graded classification of the bridge structural health assessment standard. S54. When the cumulative value of the fault exceeds the first preset threshold, a level 1 warning message is generated; when it exceeds the second preset threshold, a level 2 warning message is generated; when the cumulative value of the fault rapidly crosses the two thresholds within a preset time period, it is automatically upgraded to a level 3 warning message. Each level of warning message corresponds to a different structural damage risk level and emergency response measures. The warning message includes the abnormal fault start time, spatial location, and classification confidence level.

Images