Bridge expansion joint disease monitoring method and system based on time sequence image analysis

Abstract

The invention discloses a bridge expansion joint disease monitoring method and system based on time sequence image analysis, and belongs to the technical field of bridge structure health monitoring, and the method comprises the steps: installing fixed industrial cameras at the two sides of an expansion joint, regularly collecting high-resolution images, and synchronously obtaining temperature data; accurate registration of images in different periods is realized through feature point matching; identifying diseases such as rubber strip aging, profile steel warping, foreign matter blockage and concrete damage by adopting a deep convolutional neural network; a sub-pixel edge detection algorithm is utilized to measure the seam width, and a seam width-temperature response curve is established to identify abnormal deformation; according to the method, automatic detection, high-precision measurement and trend early warning of expansion joint diseases are realized, and the method is suitable for remote intelligent supervision of expressway bridge groups.

Description

Technical Field

[0001] This invention relates to the field of bridge structural health monitoring technology, and in particular to a method and system for monitoring bridge expansion joint defects using time-series image analysis. Background Technology

[0002] Bridge expansion joints are critical components in highway bridge structures. Their primary function is to absorb longitudinal displacements in the bridge superstructure caused by temperature changes, concrete shrinkage and creep, and vehicle loads, ensuring smooth driving and protecting the main structure from excessive internal forces. However, expansion joints operate under harsh conditions, enduring repeated impacts from vehicle wheels, rainwater erosion, temperature cycles, and debris accumulation. This makes them highly susceptible to typical defects such as aging and cracking of rubber strips, loosening and warping of steel sections, blockage of gaps by foreign objects, and damage to concrete edges. These defects not only affect driving comfort and safety but also allow rainwater to seep into bridge bearings and piers, accelerating corrosion and deterioration of the substructure, ultimately threatening the structural safety of the entire bridge.

[0003] Currently, the inspection of bridge expansion joints mainly relies on manual inspection. Inspectors need to enter the bridge deck during traffic control periods or low-traffic nighttime hours to assess the condition of the expansion joints using traditional methods such as visual observation, hammer testing, and manual measurement. This inspection method has several shortcomings: First, manual inspection cycles are usually long, making it difficult to detect the emergence and development of defects in a timely manner, especially for highway bridge groups, where on-site inspection of each bridge requires a significant amount of manpower and time. Second, the development of expansion joint defects is often insidious; early defects are often mild and difficult to accurately identify visually, and by the time defects become apparent, they have often progressed to a more serious stage. Third, manual measurement of parameters such as joint width is limited by the accuracy of measuring tools and human error, making it difficult to obtain high-precision quantitative data. Fourth, there is a lack of systematic monitoring of the relationship between expansion joints and environmental temperature response, making it impossible to distinguish between normal thermal expansion and contraction deformation and abnormal constraint failure deformation.

[0004] To address the need for automation in bridge defect detection, a series of research efforts have been conducted in related technical fields. Chinese Patent CN109300126A discloses a high-precision intelligent detection method for bridge defects based on spatial location. This method uses a drone or vehicle-mounted system equipped with a high-definition image acquisition system to collect images of various parts of the bridge. The images are then stitched together to form a panoramic image of the bridge surface. A convolutional neural network is used to train and learn from the defect images to identify defect types such as cracks. Finally, BIM and GIS technologies are combined to establish a three-dimensional spatial location archive of the defects. This technical solution has made some progress in the detection of overall bridge defects, but it still has the following limitations: First, the solution is designed for the entire bridge rather than the specific component of the expansion joint, and it fails to be specifically optimized for the special structural form and defect characteristics of the expansion joint; second, the solution uses a single image acquisition for defect identification, lacking registration and comparative analysis of images of the same part at different times, and thus cannot reveal the evolution and development patterns of the defects; third, the solution uses pixel-level resolution calculation for defect measurement, failing to achieve sub-pixel-level high-precision dimensional measurement; finally, the solution does not involve modeling the response relationship between the opening and closing amount of the expansion joint and the ambient temperature, and cannot identify abnormal deformation behavior caused by reasons such as support constraint failure.

[0005] In summary, existing technologies still have gaps in automated monitoring, temporal evolution analysis, high-precision measurement, and temperature response modeling of bridge expansion joint defects. There is an urgent need for a dedicated monitoring method and system for expansion joints to achieve timely detection, quantitative assessment, and trend warning of defects. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for monitoring bridge expansion joint defects using time-series image analysis, in order to solve the technical problems in the prior art, such as the hidden development of expansion joint defects, the difficulty in timely detection by manual inspection, the lack of high-precision quantitative measurement, and the inability to establish a temperature response model.

[0007] To achieve the above objectives, the present invention provides a method for monitoring bridge expansion joint defects using time-series image analysis, the method comprising the following steps: S1. Time-series image acquisition steps: Install fixed industrial cameras on both sides of the bridge expansion joint, and periodically acquire high-resolution images of the expansion joint area according to the preset acquisition cycle. At the same time, acquire the ambient temperature data at the acquisition time to form a time-series image sequence containing timestamps, image data and temperature data. S2. Image registration processing steps: Feature points are extracted and matched from images acquired at different times in the time-series image sequence. Based on the matched feature point pairs, geometric transformation parameters between images are calculated. Images from different times are transformed to a unified reference coordinate system to achieve precise image alignment and eliminate interference caused by changes in ambient lighting and small camera displacements. S3. Defect detection and identification steps: Input the registered image into the pre-trained defect detection network to identify the types of defects in the expansion joint area. The defect types include rubber strip aging and cracking, steel loosening and warping, foreign object blockage in the gap, and concrete edge damage. Output the location information, boundary contour and confidence level of each type of defect. S4. Dynamic monitoring steps for seam width: The sub-pixel edge detection algorithm is used to extract the precise position of the edges on both sides of the expansion joint, and the opening and closing amount of the expansion joint is calculated as the seam width measurement value. Combined with the synchronously collected ambient temperature data, a seam width-temperature response curve model is established. Abnormal deformation behavior is identified by calculating the deviation between the measured seam width and the seam width predicted by the model. S5. Trend Analysis and Early Warning Steps: Compare historical time-series image sequences to analyze the evolution trend of the area, length, and severity of various defects over time, calculate the defect deterioration rate, mark the maintenance urgency level for expansion joints whose deterioration rate exceeds the early warning threshold, and generate a bridge expansion joint health status file and maintenance recommendation report.

[0008] This invention also provides a bridge expansion joint defect monitoring system for implementing the above method, the system comprising: The time-series image acquisition module is used to install fixed industrial cameras on both sides of the bridge expansion joint, periodically acquire high-resolution images of the expansion joint area according to a preset acquisition cycle, and at the same time acquire ambient temperature data through a temperature sensor. The image registration and processing module is connected to the time-series image acquisition module and is used to extract and match feature points in images acquired at different times, calculate geometric transformation parameters, and transform the images to a unified reference coordinate system. The defect detection and identification module is connected to the image registration and processing module and is used to identify defects such as aging and cracking of rubber strips, loosening and warping of steel sections, blockage of gaps by foreign objects, and damage to concrete edges through a pre-trained defect detection network. The seam width dynamic monitoring module is connected to the image registration and processing module and the disease detection and identification module. It is used to measure the seam width using a sub-pixel edge detection algorithm, establish a seam width-temperature response curve, and identify abnormal deformation. The trend analysis and early warning module is connected to the disease detection and identification module and the crack width dynamic monitoring module. It is used to analyze the disease evolution trend, calculate the deterioration rate, mark the maintenance urgency level, and generate a health status file and maintenance recommendation report.

[0009] The beneficial effects of this invention are as follows: First, this invention establishes a dedicated monitoring method and system for bridge expansion joints, a key component. By using a fixed industrial camera, it achieves long-term continuous image acquisition, overcoming the shortcomings of traditional manual inspections, such as long cycles and poor timeliness, and enabling timely detection of the emergence and development of defects.

[0010] Secondly, this invention innovatively introduces temporal image registration technology, which precisely aligns images acquired at different times to a unified coordinate system through feature point matching and geometric transformation. This effectively eliminates interference caused by changes in ambient lighting and minute camera displacements, laying a reliable data foundation for subsequent disease comparison and analysis.

[0011] Third, the present invention uses a sub-pixel edge detection algorithm to achieve high-precision measurement of the opening and closing of expansion joints, with a measurement accuracy of up to 0.02mm, which is far superior to traditional manual measurement methods and can capture the minute deformation of expansion joints.

[0012] Fourth, this invention innovatively establishes a joint width-temperature response curve model. By analyzing the correspondence between joint width and ambient temperature, it identifies abnormal deformation behavior and can distinguish between normal thermal expansion and contraction deformation and abnormal deformation caused by support constraint failure, anchor loosening, etc., providing an important basis for structural safety assessment.

[0013] Fifth, by analyzing the evolution trend of disease characteristics in historical time-series image sequences, calculating the deterioration rate, and establishing a maintenance urgency classification system, this invention realizes the transformation from passive detection to proactive early warning, providing scientific support for the preventive maintenance decisions of bridge maintenance management departments. Attached Figure Description

[0014] Figure 1 This is a flowchart of the bridge expansion joint defect monitoring method based on time-series image analysis described in this invention.

[0015] Figure 2 This is an architecture diagram of the bridge expansion joint defect monitoring system described in this invention. Detailed Implementation

[0016] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0017] See Figure 1 The bridge expansion joint defect monitoring method based on time-series image analysis described in this invention includes five core steps, which form a deeply coupled closed-loop collaborative architecture: the output of step S1 serves as the key input of step S2, the output of step S2 serves as the input of steps S3 and S4, the outputs of steps S3 and S4 serve as the input of step S5, and the trend analysis results of step S5 can be fed back to adjust the acquisition cycle of step S1 and the early warning threshold of step S4, forming a complete closed-loop control.

[0018] Step S1: Time-series image acquisition step.

[0019] The time-series image acquisition step of this invention aims to establish a long-term, continuous monitoring capability for bridge expansion joints, acquiring high-quality image data and environmental parameter data to provide a reliable data source for subsequent registration processing, defect detection, and trend analysis. This step is the data entry point for the entire monitoring method, and its data quality directly affects the accuracy and reliability of all subsequent processing steps. Therefore, meticulous design is required in all aspects, including hardware selection, installation layout, parameter configuration, and data management. The core concept of time-series image acquisition lies in capturing subtle changes in the state of expansion joints over time through long-term, continuous, fixed-point observations, which is difficult to achieve using traditional manual inspection methods.

[0020] In a preferred embodiment of the present invention, the hardware configuration of the time-series image acquisition system needs to comprehensively consider factors such as resolution, sensitivity, dynamic range, and environmental adaptability. Regarding camera selection, a fixed industrial camera with an industrial-grade CMOS image sensor is used, preferably a back-illuminated structure, which has higher quantum efficiency and lower dark current noise. The resolution is selected to be at least 20 megapixels, preferably 24 megapixels, with a 1-inch sensor size and a pixel size of 3.45μm × 3.45μm, ensuring that at a typical shooting distance of 1.5m, the actual size of a single pixel is approximately 0.1mm, meeting the resolution requirements for expansion joint defect detection. Regarding dynamic range, the camera's dynamic range is required to be no less than 72dB to accommodate the significant brightness difference between the metal and concrete surfaces in the expansion joint area. Regarding signal-to-noise ratio (SNR), the SNR under standard gain conditions is no less than 45dB, ensuring clear images can still be obtained under low-light conditions. Regarding frame rate, while static image acquisition does not require a high frame rate, a frame rate of no less than 20fps is necessary to ensure shutter synchronization and reduce motion blur. The camera lens is a fixed-focus industrial lens with a focal length of 25mm and an aperture range of F1.4 to F16, featuring automatic aperture adjustment to adapt to different lighting conditions. The lens distortion rate is controlled within 0.5% to ensure that the image geometric accuracy meets measurement requirements. In one embodiment of the invention, the lens employs a multi-layer coating process to reduce stray light and ghosting, improving image contrast and sharpness.

[0021] The choice of camera installation location is crucial for obtaining high-quality images. Preferably, the camera is mounted on the bridge deck railing or a dedicated bracket on both sides of the expansion joint, with the camera's optical axis perpendicular to the bridge deck and pointing towards the expansion joint area. The shooting distance is controlled between 1.2m and 2.0m. In one embodiment of the invention, a dual-camera symmetrical arrangement is adopted, with two cameras located on either side of the expansion joint, each covering half of the expansion joint area with appropriate overlap. The overlap area accounts for 15% to 25% of the width of a single image, facilitating subsequent image stitching. The camera is protected by an IP67-rated protective shell, with a built-in temperature-controlled heating device and a defogging system, ensuring stable operation within an ambient temperature range of -30℃ to 60℃ and in rainy or foggy weather conditions.

[0022] The image acquisition cycle setting needs to comprehensively consider factors such as the disease development speed, temperature change cycle, and data storage capacity. In one embodiment of the present invention, the acquisition cycle is set to an adaptive mode: under normal monitoring conditions, the acquisition cycle is 24 hours, that is, images are acquired once a day at the same time; when an abnormality is detected or a key observation period is entered, the acquisition cycle is automatically adjusted to 1 to 6 hours; during seasonal transitions with drastic temperature changes, the acquisition cycle is adjusted to 4 hours to capture the dynamic response process of the crack width. Each time an image is acquired, the system automatically records the timestamp of the acquisition time with an accuracy of seconds.

[0023] Synchronous acquisition of ambient temperature data plays a crucial role in establishing the joint width-temperature response curve. In one embodiment of the present invention, a digital temperature sensor is installed near the expansion joint, with a measurement range of -40℃ to 85℃, a measurement accuracy of ±0.2℃, and a resolution of 0.1℃. The temperature sensor is installed 30cm to 50cm below the bridge deck from the edge of the expansion joint, protected by a sunshade to reduce the influence of direct solar radiation, ensuring that the measured value represents the actual ambient temperature at the location of the expansion joint. The temperature data sampling frequency is set to once per minute, and timestamps are matched with the image acquisition time to ensure synchronization between the temperature data and the image data.

[0024] Illumination compensation during image acquisition is a crucial step in ensuring consistent image quality. In one embodiment of this invention, the camera system is equipped with a ring-shaped LED fill light with a color temperature of 5600K and a power of 24W, providing uniform illumination at night or on cloudy days. The brightness of the fill light is automatically adjusted according to the ambient light intensity; it automatically turns on when the ambient illuminance is below 200 lux and automatically turns off when the illuminance is above 500 lux. Preferably, a polarizing filter is used to eliminate specular reflections from water accumulation on bridge surfaces and metal surfaces of expansion joints, improving image contrast and detail clarity.

[0025] Regarding data transmission and storage, in one embodiment of this invention, the collected image and temperature data are transmitted to a data center server via a 4G / 5G wireless network or wired Ethernet. The original images are stored in a lossless compression format, with each image file being approximately 12MB to 18MB in size. To facilitate long-term archiving and rapid retrieval, a tiered storage strategy is established: data from the most recent 30 days is stored in a high-speed solid-state drive array, data from 30 days to one year is stored in a mechanical hard drive array, and historical data older than one year is archived to a tape library or cloud storage. The annual data volume for each bridge expansion joint is approximately 6GB to 20GB, requiring the data center to reserve sufficient storage space.

[0026] Step S2: Image registration process.

[0027] Image registration is the core technical step in this invention for achieving temporal image comparison and analysis. Due to slight positional drift that may occur after camera installation, coupled with differences in lighting conditions at different times of image acquisition, direct pixel-level comparison would produce numerous spurious differences. This step achieves precise alignment of images from different periods through feature point matching and geometric transformation, providing a reliable data foundation for subsequent disease detection and crack width measurement.

[0028] In a preferred embodiment of the present invention, the image registration process includes the following sub-steps: First, image preprocessing is performed to improve the stability of feature point extraction. The original image is converted to grayscale, transforming the RGB three-channel image into a single-channel grayscale image. In one embodiment of the invention, the grayscale conversion formula uses a weighted average method, with a weight of 0.299 for the red channel, 0.587 for the green channel, and 0.114 for the blue channel, conforming to the characteristics of human visual perception. Subsequently, the grayscale image undergoes contrast enhancement processing using an adaptive histogram equalization algorithm. The image is divided into 8×8 sub-blocks, and histogram equalization is performed on each sub-block before interpolation and fusion, avoiding noise amplification caused by over-enhancement.

[0029] Secondly, feature point extraction is performed. This invention preferably uses the SIFT algorithm for feature point extraction. The SIFT algorithm has good invariance to image rotation, scaling, and brightness changes, making it suitable for processing images acquired at different times. In one embodiment of this invention, the parameters of the SIFT algorithm are set as follows: the Gaussian pyramid has 4 groups, each group has 5 layers, the initial standard deviation σ of the Gaussian filter is 1.6, and the scale factor k between adjacent layers is 2 to the power of 1 / 3. The contrast threshold for feature point detection is set to 0.04, and the edge response threshold is set to 10 to remove low-contrast points and edge response points. Typically, 3,000 to 8,000 SIFT feature points can be extracted from a single 24-megapixel expansion joint image.

[0030] As an alternative, another embodiment of the present invention employs the ORB algorithm for feature point extraction. The ORB algorithm has higher computational efficiency than the SIFT algorithm and is suitable for resource-constrained embedded platforms. The parameters of the ORB algorithm are set as follows: the maximum number of feature points is 5000, the number of pyramid layers is 8, the scaling factor is 1.2, the threshold for FAST corner detection is 20, and the edge threshold is 31. The ORB algorithm uses a rotation-invariant version of the BRIEF descriptor, with the descriptor dimension being a 256-bit binary vector.

[0031] Next, feature point matching is performed. In one embodiment of the invention, a brute-force matching algorithm combined with a cross-validation strategy is used for feature point matching. For each feature point in the reference image, the feature point with the closest descriptor distance in the image to be registered is searched as a candidate matching point. For SIFT descriptors, Euclidean distance is used as the distance metric; for ORB descriptors, Hamming distance is used. To improve matching accuracy, a ratio test strategy is adopted, that is, the ratio of the nearest neighbor distance to the second nearest neighbor distance must be less than a threshold of 0.75 to retain the matching pair. At the same time, a cross-validation strategy is adopted, and only feature point pairs selected in both forward and reverse matching are considered valid matches. Typically, the number of valid matching points after the above screening is 500 to 2000 pairs.

[0032] Then, robust estimation of the geometric transformation matrix is ​​performed. In one embodiment of the present invention, the RANSAC algorithm is used for robust estimation of the transformation matrix to eliminate the influence of mismatched points. The parameters of the RANSAC algorithm are set as follows: the upper limit of the number of iterations is 2000, the inlier threshold is 3.0 pixels, and the confidence level is 0.99. In each iteration, four pairs of matching points are randomly selected to calculate the perspective transformation matrix (or three pairs of matching points are selected to calculate the affine transformation matrix). The number of points among all matching points that satisfy the reprojection error being less than the inlier threshold is counted as the support of the transformation matrix. The transformation matrix with the highest support is selected as the final result, and least squares optimization is performed again using all inliers to obtain more accurate transformation parameters.

[0033] The choice of geometric transformation model is determined based on the stability and accuracy requirements of the camera installation. In a preferred embodiment of the invention, when the camera installation is stable and only has minor displacement, an affine transformation model is used. The affine transformation matrix contains 6 degrees of freedom and can describe translation, rotation, scaling, and shear transformations. The general form of the affine transformation matrix is ​​a 3×3 matrix, where the third row is fixed at [0, 0, 1]. In another embodiment of the invention, when the camera may experience significant changes in viewing angle, a perspective transformation model is used. The perspective transformation matrix contains 8 degrees of freedom and can describe projection transformations between arbitrary planes. The general form of the perspective transformation matrix is ​​a 3×3 matrix, and all elements can be varied.

[0034] Finally, image resampling is performed to generate the registered image. In one embodiment of the present invention, a bicubic interpolation algorithm is used for image resampling. Bicubic interpolation considers the gray values ​​of 16 pixels in a 4×4 neighborhood around the target pixel, and a weighted average is performed using a cubic polynomial function to obtain a smooth interpolation result. The registered image maintains the same size and resolution as the reference image, and the registration accuracy can reach 0.3 to 0.5 pixels.

[0035] To verify the registration quality, in one embodiment of the present invention, normalized mutual information is used as an evaluation index for registration accuracy. The value of normalized mutual information ranges from 0 to 2, with a larger value indicating a better degree of registration between the two images. When the normalized mutual information is greater than 1.5, the registration result is considered acceptable; when the normalized mutual information is less than 1.2, the system issues a registration anomaly warning, indicating that there may be significant camera displacement or image quality issues, requiring manual intervention for inspection.

[0036] Step S3: Disease detection and identification steps.

[0037] Defect detection and identification is a key step in the expansion joint condition assessment of this invention. This step uses a pre-trained deep convolutional neural network to analyze the registered images, automatically identifying typical defect types such as aging and cracking of rubber strips, loosening and warping of steel sections, blockage of gaps by foreign objects, and damage to concrete edges, and outputting the location, boundary, and confidence information of the defects.

[0038] In a preferred embodiment of the present invention, the disease detection network adopts an attention-based target detection architecture, and the overall structure includes three main parts: a feature extraction backbone network, a multi-scale feature fusion module, and a detection head module.

[0039] A feature extraction backbone network is used to extract multi-level semantic features from the input image. In one embodiment of the present invention, the backbone network adopts a ResNet-50 structure, comprising one convolutional layer, four residual module groups, and one global average pooling layer. The first convolutional layer has a kernel size of 7×7, a stride of 2, and 64 output channels. The four residual module groups contain 3, 4, 6, and 3 residual blocks, respectively, with output channels of 256, 512, 1024, and 2048, respectively, and spatial resolution decreasing to 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input, respectively. Each residual block adopts a bottleneck structure, comprising three convolutional layers of 1×1, 3×3, and 1×1 and skip connections. The weights of the backbone network are initialized using parameters pre-trained on the ImageNet dataset to accelerate convergence and improve generalization ability.

[0040] A multi-scale feature fusion module is used to integrate semantic features at different levels, enabling the network to detect both large-scale and small-scale diseases simultaneously. In one embodiment of the invention, a feature pyramid network structure is adopted, fusing high-level semantic features with low-level detail features through top-down paths and lateral connections. Specifically, lateral connections are drawn from layers 2, 3, 4, and 5 of the backbone network. High-level features are upsampled by a factor of 2 and then element-wise added to low-level features. The fused features are then smoothed using a 3×3 convolution. After fusion, feature maps at four scales are generated, with spatial resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input, respectively, and a uniform number of channels of 256.

[0041] To enhance the network's ability to focus on diseased areas, in one embodiment of this invention, a channel attention mechanism and a spatial attention mechanism are introduced after the feature fusion module. The channel attention module first performs global average pooling on the feature map to obtain a global descriptive vector along the channel dimension. Then, it generates channel attention weights through a two-layer fully connected network. The first fully connected layer compresses the number of channels to 1 / 16 of the original number using ReLU activation, and the second fully connected layer restores the number of channels using Sigmoid activation. The spatial attention module first calculates the maximum and average values ​​along the channel dimension to obtain two single-channel feature maps. These two maps are concatenated and then processed through a 7×7 convolution and Sigmoid activation to generate a spatial attention weight map. The final attention feature is the original feature multiplied by the channel attention weights and the spatial attention weights consecutively.

[0042] The detection head module is responsible for predicting the category and location of diseases. In one embodiment of the invention, the detection head adopts a decoupled design, with the classification branch and regression branch operating independently. The classification branch contains four 3×3 convolutional layers, each followed by batch normalization and ReLU activation, and finally outputs the category confidence score through a 1×1 convolution, with the number of output channels equal to the number of disease categories, which is 4. The regression branch also contains four 3×3 convolutional layers, and finally outputs the position offset of the bounding box through a 1×1 convolution, with 4 output channels corresponding to the x and y offsets of the bounding box center point and the scaling factors for width and height, respectively. The detection head operates independently on the feature maps at each scale, and finally merges the multi-scale detection results using non-maximum suppression.

[0043] The training dataset for the disease detection network needs to cover various typical diseases. In one embodiment of the present invention, the training dataset contains 5000 expansion joint images, including 1800 samples of aging and cracking rubber strips, 1200 samples of loose and warped steel sections, 1500 samples of foreign object blockage in the joints, 1500 samples of damaged concrete edges, and 2000 normal samples without diseases. Image annotation uses bounding box annotation, with each diseased region labeled with its minimum bounding rectangle and category label. Data augmentation strategies include random horizontal flipping, random rotation ±15°, random brightness adjustment ±20%, and random contrast adjustment ±20% to improve the model's generalization ability.

[0044] The network was trained using the SGD optimizer with an initial learning rate of 0.01, momentum of 0.9, and weight decay of 0.0001. Cosine annealing was used for learning rate scheduling, and the total number of training epochs was 100. The loss function consisted of two parts: classification loss and regression loss. Focal Loss was used for classification to mitigate the imbalance between positive and negative samples, and Smooth L1 Loss was used for regression. Training was performed on a server equipped with an NVIDIA RTX 3090 graphics card, with a batch size of 16, and each training epoch took approximately 2 hours.

[0045] During the inference process for disease detection, the input image size is adjusted to 512×512 pixels or 1024×1024 pixels. A larger input size can obtain more refined detection results, but the inference time also increases accordingly. In one embodiment of the present invention, when using an input size of 1024×1024, the inference time for a single image is approximately 50ms to 80ms. The confidence threshold is set to 0.5 to 0.8. A higher threshold results in higher confidence in the detected diseases but may miss some minor diseases, while a lower threshold increases the recall rate but may introduce false positives. In one embodiment of the present invention, the confidence threshold is set to 0.6 by default, and users can adjust it according to actual needs. The IoU threshold for non-maximum suppression is set to 0.5 to merge overlapping detection boxes.

[0046] To address the unique characteristics of expansion joint defect detection, this invention incorporates several targeted optimization measures during network training and inference. First, considering that expansion joint defects typically exhibit a long and narrow shape, such as cracks in rubber strips and gaps in steel sections, aspect ratio transformation is added during the data augmentation stage. The standard square cropping is expanded into rectangular cropping with an aspect ratio of 1:3 to 3:1, enabling the network to better adapt to the detection of long and narrow targets. Second, considering the textural differences between metal and concrete materials in expansion joint images, CLAHE adaptive histogram equalization is added during the preprocessing stage to enhance the visibility of details in low-contrast areas. Third, for defects such as loose and warped steel sections that require assessment of three-dimensional deformation, a multi-view fusion strategy is introduced. This improves detection reliability by analyzing images of the same expansion joint taken from different angles.

[0047] Post-processing of the detection results is a crucial step in improving the final output quality. In one embodiment of this invention, temporal consistency filtering is performed on the original detection results. This involves correlation analysis of the detection results across multiple consecutive frames, confirming only defects detected in at least three consecutive frames as genuine defects, thus filtering out occasional false detections. Simultaneously, the defect bounding boxes are smoothed, and Kalman filtering is used to track and predict defect locations, reducing bounding box jitter caused by image noise or lighting variations.

[0048] The output of the detection results includes: disease category, bounding box coordinates (x and y coordinates of the top left corner, as well as width and height), and confidence score. For ease of subsequent analysis, the detection results are stored in JSON format. Each record contains an image ID, timestamp, and a list of diseases. Each disease entry contains a category code, bounding box coordinates, and confidence score.

[0049] Step S4: Dynamic monitoring of seam width.

[0050] Dynamic monitoring of expansion joint width is one of the core innovative steps of this invention, aiming to achieve high-precision measurement of the opening and closing amount of expansion joints and establish a response relationship model between joint width and temperature. This step uses a sub-pixel edge detection algorithm to overcome the accuracy limitations of traditional pixel-level measurement, and simultaneously identifies abnormal deformation behavior through response curve modeling.

[0051] In a preferred embodiment of the present invention, the processing flow of the sub-pixel edge detection algorithm includes the following sub-steps: First, the edge region of the expansion joint is located and extracted. Based on the defect detection results in step S3 or the preset expansion joint location information, a region of interest (ROI) containing both sides of the expansion joint is cropped from the registered image. In one embodiment of the present invention, the width of the ROI is set to 3 to 5 times the estimated width of the expansion joint, and the height is set to 50 to 200 pixels to ensure that both sides are fully included and sufficient edge detection buffer area is provided.

[0052] Secondly, multi-scale Gaussian filtering preprocessing is performed to suppress image noise. In one embodiment of the present invention, scale-space theory is used for multi-scale filtering, with the standard deviation σ of the Gaussian kernel set to three scales: 0.5, 1.0, and 2.0. Small-scale filtering retains more details but is sensitive to noise, while large-scale filtering has a good smoothing effect but may blur edges. The final filtering result adopts a multi-scale fusion strategy, adaptively selecting the optimal scale based on the local gradient strength.

[0053] Then, gradient calculation is performed to detect edge locations. In one embodiment of the invention, the Sobel operator is used to calculate the gradient components of the image in the horizontal and vertical directions. The horizontal Sobel operator is a 3×3 matrix with coefficients distributed as [[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]]; the vertical Sobel operator is also a 3×3 matrix with coefficients distributed as [[-1,-2, -1], [0, 0, 0], [1, 2, 1]]. The gradient magnitude is calculated as the Euclidean norm of the horizontal and vertical gradients, and the gradient direction is calculated as the arctangent of the ratio of the vertical gradient to the horizontal gradient.

[0054] Since the edges of expansion joints typically extend longitudinally along the bridge, their edge direction is nearly horizontal or nearly vertical. In one embodiment of the present invention, the main gradient direction is determined based on the orientation of the expansion joint, and only pixels whose gradient direction forms an angle of less than 30° with the normal direction of the expansion joint edge are retained as candidate edge points.

[0055] Next, non-maximum suppression is performed to obtain edges with a single pixel width. The gradient magnitude of each candidate edge point is checked along the gradient direction to see if it is a local maximum. If the gradient magnitude of that point is greater than the gradient magnitudes of its two adjacent pixels along the gradient direction, the point is retained; otherwise, it is suppressed. Non-maximum suppression yields edge positions with pixel-level precision.

[0056] Finally, the precise sub-pixel edge position is calculated. This invention innovatively proposes a gradient-weighted centroid method, which calculates the weighted centroid as the sub-pixel edge coordinates within the local neighborhood of a pixel-level edge point based on the gradient magnitude.

[0057] The mathematical expression of the gradient-weighted centroid method is as follows: Let the coordinates of the pixel-level edge points be... In its Sub-pixel edge coordinates within the neighborhood (N is 5 in one embodiment of the invention). The calculation formula is: , ,in, These are the gradient weighting coefficients, and their calculation formula is as follows: ,in: For pixels The gradient magnitude at the specified location, expressed in grayscale values ​​per pixel. The standard deviation of the spatial Gaussian kernel is 1.5 pixels in one embodiment of the present invention, and is used to control the spatial attenuation weight of neighboring pixels; For pixels The gradient direction at the location, in radians; pixel-level edge points The gradient direction at the location, in radians; The gradient direction consistency threshold is set to a value in one embodiment of the present invention. Radius, or 30°; This is an indicator function. It takes the value 1 when the difference in gradient direction is less than the threshold, and 0 otherwise. It is used to remove noise points whose gradient direction is inconsistent with the main edge direction.

[0058] The advantages of the gradient-weighted centroid method are: by weighting the gradient magnitude, pixels with stronger edge responses contribute more to the sub-pixel position, improving the robustness of edge localization; by using spatial Gaussian weighting, pixels closer to the pixel-level edge contribute more, avoiding interference from distant noise; and by constraining the gradient direction consistency, noise responses inconsistent with the main edge direction are eliminated, further improving localization accuracy. The sub-pixel edge detection algorithm of this invention achieves a localization accuracy of 0.1 to 0.02 pixels on measured data, corresponding to an actual size accuracy of approximately 0.01 mm to 0.002 mm.

[0059] After completing the sub-pixel localization of the two side edges, the width of the expansion joint is calculated. In one embodiment of the invention, a measurement point is selected every 10 pixels along the longitudinal direction of the expansion joint, and the vertical distance between the two side edges is calculated at each measurement point as the width value of that point. Median filtering is applied to the width values ​​of all measurement points to remove outliers, and the median is finally taken as the width measurement result of the current image. .

[0060] Modeling the joint width-temperature response curve is another core innovation of this invention. Based on thermodynamic principles, the longitudinal displacement of the bridge superstructure has an approximately linear relationship with temperature changes, and the opening and closing amount of the expansion joint reflects this displacement change. In one embodiment of this invention, a linear model is used to describe the response relationship between joint width and temperature: ,in: for The seam width measurement at any given time, in mm; The reference seam width at the reference temperature is expressed in mm and typically ranges from 20 mm to 100 mm. for The ambient temperature at any given time, in °C; The reference temperature is 20°C in one embodiment of the present invention; The dynamic response coefficient is expressed in mm / ℃. Physically, it represents the change in joint width when the temperature changes by 1℃. The value typically ranges from 0.5 mm / ℃ to 3.0 mm / ℃, and the specific value depends on factors such as bridge span, structural form, and bearing type.

[0061] The parameters of the response curve are estimated using the least squares method. In one embodiment of the invention, at least 30 days of synchronous monitoring data on suture width and temperature are collected, and linear regression is performed with suture width as the dependent variable and temperature as the independent variable to obtain the dynamic response coefficients. and reference seam width The estimated value. Goodness of fit of the regression model. It should generally be greater than 0.85. A value below 0.7 indicates a possible nonlinear response or data quality issue, requiring further analysis.

[0062] For cases with large temperature variations or nonlinear response relationships, another embodiment of the present invention employs a piecewise linear model: .in: This is the temperature segmentation threshold, in °C. In one embodiment of the present invention, the value is 35 °C. The value represents the dynamic response coefficient in the low-temperature range, expressed in mm / ℃. The dynamic response coefficient for the high-temperature range is expressed in mm / ℃. The segmented model can adapt to the changes in the response characteristics of the expansion joint during high summer temperatures.

[0063] The identification of abnormal deformation behavior is based on the deviation analysis between the measured seam width and the seam width predicted by the response curve. Seam width deviation is defined. for: ,in: for The measured seam width at any given time, in mm; The predicted seam width is calculated based on the response curve model and the current temperature, in mm.

[0064] Under normal circumstances, the seam width deviation should follow a normal distribution with a mean of 0 and a standard deviation of [missing value]. This reflects the combined effects of measurement noise and model error. In one embodiment of the present invention, the criterion for determining abnormal deformation behavior is: When the joint width deviation exceeds three times the standard deviation, it is considered abnormal deformation, and the system issues a warning. Possible causes of abnormal deformation include: support constraint failure leading to substructure displacement exceeding the normal range, loose expansion joint anchorage causing overall steel displacement, and foreign object blockage obstructing the opening and closing of the expansion joint. Upon detecting abnormal deformation, the system automatically increases the image acquisition frequency and triggers a manual review process.

[0065] Step S5: Trend Analysis and Early Warning Step.

[0066] Trend analysis and early warning are key steps in achieving preventative maintenance in this invention. This step analyzes the evolution trend of defect characteristics in historical time-series image sequences, calculates the deterioration rate, and establishes a maintenance urgency classification system to provide decision support for bridge maintenance management.

[0067] In a preferred embodiment of the present invention, the method for calculating the disease deterioration rate is as follows: First, define the characteristic quantities of the disease. In one embodiment of the present invention, corresponding characteristic quantities are defined for different disease types: the characteristic quantity of rubber strip aging and cracking is the total length of the crack. (Unit: mm) and maximum crack width (Unit: mm); The characteristic measure of loosening and warping of structural steel is the warping height. (Unit: mm); The characteristic quantity of foreign object blockage in gaps is the blockage area ratio. (Unit: %); The characteristic quantity of concrete edge damage is the damaged area. (Unit: mm) 2 ).

[0068] Secondly, calculate the changes in characteristic quantities within adjacent monitoring periods. Let... and For two adjacent monitoring times, and Let each be a characteristic value at the corresponding time point, and the change in the characteristic value is... for: Then, the degradation rate is calculated. In one embodiment of the invention, the degradation rate is... Defined as the change in the characteristic value divided by the monitoring time interval, and normalized to a monthly rate of change: ,in: The degradation rate is expressed as a percentage per month. This represents the change in the characteristic quantity, with the same unit as the characteristic quantity. As a reference benchmark value for the feature quantity, in one embodiment of the present invention, the feature quantity value at the initial detection or the allowable value specified by industry standards is taken; The monitoring time interval is in days; 30 is the normalization factor, which converts the daily rate of change to the monthly rate of change; 100% is the percentage conversion factor.

[0069] To improve the stability of degradation rate estimation, in one embodiment of the present invention, a sliding window method is used to calculate the average degradation rate. Let the length of the sliding window be... One monitoring cycle (in one embodiment of the present invention) (If the value is between 5 and 10), then the average degradation rate is... This is the arithmetic mean of the degradation rates over each period within the window. The sliding window method can smooth out short-term fluctuations and reveal the overall trend of disease development.

[0070] The maintenance urgency classification is an intelligent decision-making system based on the rate of degradation. In one embodiment of the present invention, maintenance urgency is divided into four levels: Level 1 Emergency Repair: When the average degradation rate Exceeding the first warning threshold (In one embodiment of the present invention, the value is 10% / month), or when structural damage is detected (such as steel section fracture or anchorage failure), an emergency maintenance warning is issued immediately, and it is recommended to arrange on-site inspection and maintenance within 7 days.

[0071] Secondary priority maintenance: when the average degradation rate Between the second warning threshold (In one embodiment of the present invention, the value is 5% / month) and the first warning threshold. If this occurs, a priority maintenance warning will be issued, and maintenance will be arranged within 30 days.

[0072] Level 3 Planned Maintenance: When the average degradation rate Between the third warning threshold (In one embodiment of the present invention, the value is 2% / month) and the second early warning threshold. During this period, a planned maintenance reminder will be issued, suggesting that it be included in the quarterly or annual maintenance plan.

[0073] Level 4 observation and monitoring: When the average degradation rate Below the third warning threshold At that time, the expansion joint was in good condition, and routine monitoring continued.

[0074] The aforementioned warning thresholds can be adjusted based on factors such as the importance level of the expansion joint, traffic flow, and the location of the bridge. In one embodiment of the present invention, a lower threshold is used for mainline bridges of expressways to increase the safety margin, while the threshold can be appropriately relaxed for low-grade highway bridges to optimize the allocation of maintenance resources.

[0075] The health status file of bridge expansion joints includes: basic information of the expansion joint (bridge number, expansion joint type, installation date, design parameters), historical image sequence (all acquired images and metadata stored in chronological order), defect detection records (defect type, location, and confidence level for each detection), joint width monitoring data (joint width time series curve, temperature time series curve), temperature response curve parameters (dynamic response coefficient, reference joint width, goodness of fit), deterioration trend curve (curves showing the change of each characteristic quantity over time, deterioration rate curve), and maintenance recommendations (current urgency level, recommended maintenance measures, and expected maintenance time window).

[0076] The maintenance recommendation report is generated automatically and includes: statistics on the distribution of disease types (the number, proportion, and typical examples of each type of disease), risk assessment of deterioration (risk level assessment based on the rate of deterioration and analysis of potential safety hazards), priority ranking of maintenance (ranking multiple expansion joints according to their urgency level), and maintenance measures recommendations (providing specific maintenance solutions for different types of diseases).

[0077] The closed-loop control mechanism is a key feature of this invention. The trend analysis results in step S5 can be used to adjust the acquisition cycle in step S1: when the degradation rate is high, the acquisition cycle is automatically shortened to achieve more intensive monitoring; when the state is stable, the acquisition cycle can be appropriately extended to save system resources. Furthermore, the detection results of abnormal deformation can dynamically adjust the warning threshold in step S4, establishing an adaptive anomaly identification mechanism.

[0078] This invention also provides various visualization methods to assist in maintenance management decisions. In one embodiment of this invention, the system provides the following visualization functions: a heat map of disease distribution, with the longitudinal position of the expansion joint as the horizontal axis and monitoring time as the vertical axis, using different shades of color to represent the density of diseases at each location, intuitively displaying the spatiotemporal distribution pattern of diseases; a scatter plot of joint width and temperature, with the ambient temperature on the horizontal axis and the measured joint width on the vertical axis, overlaid with a response curve fitting line, facilitating the identification of abnormal data points deviating from the normal response; a deterioration trend curve, with time as the horizontal axis and characteristic quantity as the vertical axis, plotting the time evolution curve of disease characteristic quantity and its predictive extension line, assisting in judging the future development trend of diseases; a pie chart of urgency distribution, statistically analyzing the proportion of expansion joints at each level according to urgency, facilitating the grasp of the overall maintenance workload; and a maintenance timeline, with time as the horizontal axis, displaying the suggested maintenance window period and actual maintenance records for each expansion joint, supporting the formulation and tracking of maintenance plans.

[0079] To improve the accuracy of trend prediction, one embodiment of this invention introduces a machine learning method for disease development prediction. Specifically, a time series prediction model is established based on historical monitoring data, and a long short-term memory network is used to capture the long-term dependencies in disease evolution. The model input is a sequence of feature quantities from the past N monitoring periods, and the output is the predicted value of the feature quantities for the next M monitoring periods. In one embodiment of this invention, N is set to 30, meaning data from the past 30 periods is used, and M is set to 12, meaning the trend for the next 12 periods is predicted. The model training uses the first 80% of the historical data as the training set and the last 20% as the validation set. The loss function is the mean squared error, and the optimizer is the Adam algorithm. Verification shows that the prediction model's average prediction error on a one-month timescale is controlled within 15%, providing valuable reference for maintenance planning.

[0080] Furthermore, this invention also considers the impact of seasonal factors on disease development. In most parts of my country, the effects of low winter temperatures and high summer temperatures on expansion joints differ significantly: in winter, expansion joints are at their maximum opening, making them prone to rubber strip tearing; in summer, expansion joints are at their minimum opening, making them prone to blockage by foreign objects. In one embodiment of this invention, a seasonal decomposition method is introduced into the trend analysis, decomposing the time series of disease characteristics into trend terms, seasonal terms, and residual terms, which are then analyzed and predicted separately. The trend term reflects the long-term deterioration pattern of the disease, the seasonal term reflects the periodic changes of the disease with the seasons, and the residual term reflects random fluctuations and sudden events. This decomposition method helps to more accurately identify the essential development trend of the disease and avoids interference from seasonal fluctuations.

[0081] See Figure 2 The bridge expansion joint defect monitoring system of the present invention includes a time-series image acquisition module, an image registration and processing module, a defect detection and identification module, a joint width dynamic monitoring module, and a trend analysis and early warning module, and the modules form an organically coordinated overall architecture.

[0082] The time-series image acquisition module is located at the bridge site and mainly includes an industrial camera unit, a temperature acquisition unit, a data transmission unit, and a local control unit. The industrial camera unit is responsible for periodically acquiring high-resolution images of the expansion joint area. In one embodiment of this invention, the camera uses a Sony IMX183 sensor with 20 million effective pixels, a frame rate of 20fps, and a GigE Ethernet interface. The temperature acquisition unit is responsible for synchronously acquiring ambient temperature data. In one embodiment of this invention, a Pt100 platinum resistance temperature sensor is used in conjunction with a 24-bit ADC converter, achieving a measurement accuracy of ±0.1℃. The data transmission unit is responsible for transmitting the acquired image data and temperature data to the data center. In one embodiment of this invention, an industrial 4G router is used for wireless transmission, supporting a VPN encrypted tunnel to ensure data security. The local control unit is responsible for coordinating the working sequence of each acquisition unit. In one embodiment of this invention, an embedded controller with an ARM Cortex-A72 processor is used, running a Linux operating system and customized acquisition control software.

[0083] The image registration processing module resides on a data center server and is responsible for registering and aligning the received time-series images. In one embodiment of the invention, the registration processing module is deployed on a server configured with an Intel Xeon processor and 64GB of memory, and uses the OpenCV library to implement SIFT feature extraction, feature matching, and RANSAC transform estimation algorithms. The registration processing supports batch mode and real-time mode: batch mode is used to process historically accumulated image data, while real-time mode is used to register newly acquired images instantly. The registered images are stored on a high-speed NVMe solid-state drive array, supporting fast random access.

[0084] The disease detection and identification module, also located in the data center, is responsible for identifying disease types in the registered images. In one embodiment of this invention, the disease detection module is deployed on a GPU server equipped with an NVIDIA RTX 4090 graphics card, and the inference of the disease detection network is implemented using the PyTorch deep learning framework. The detection module provides a REST API interface, supporting remote calls and batch processing. Detection results are stored in a PostgreSQL relational database for easy subsequent querying and statistical analysis.

[0085] The seam width dynamic monitoring module works closely with the image registration and processing module and the disease detection and identification module to achieve sub-pixel-level seam width measurement and temperature response curve modeling. In one embodiment of the invention, the seam width monitoring module employs a self-developed gradient-weighted sub-pixel edge detection algorithm, implemented in C++ and optimized using SIMD vectorization, achieving a seam width calculation time of less than 100ms for a single image. Response curve modeling is implemented using the SciPy library in Python, supporting parameter estimation for both linear and piecewise linear models. Seam width time-series data and response curve parameters are stored in the time-series database InfluxDB, supporting efficient time-range querying and aggregation calculations.

[0086] The trend analysis and early warning module is the decision-making center of the entire system, responsible for comprehensively analyzing disease detection results and crack width monitoring data to generate health status files and maintenance recommendation reports. In one embodiment of the invention, the trend analysis module is developed using Python, calling the Pandas library for data processing, the Matplotlib library for trend curve binding, and the ReportLab library to generate PDF-format maintenance recommendation reports. Early warning messages are notified to relevant management personnel through multiple channels such as email, SMS, and mobile app push notifications.

[0087] The data flow between modules is as follows: The time-series image acquisition module transmits raw image data and temperature data to the image registration processing module; the image registration processing module transmits the registered image to both the disease detection and identification module and the joint width dynamic monitoring module; the disease detection and identification module transmits the disease detection results to the trend analysis and early warning module and the joint width dynamic monitoring module (used to locate the edge area of ​​the expansion joint); the joint width dynamic monitoring module transmits the joint width measurement data and response curve parameters to the trend analysis and early warning module; the trend analysis and early warning module integrates all input information to generate a health status file and maintenance suggestion report, and feeds back the acquisition cycle adjustment instruction to the time-series image acquisition module, forming a complete closed-loop control.

[0088] The system's communication protocol design considers the comprehensive requirements of real-time performance, reliability, and security. In one embodiment of this invention, the field acquisition device and the data center communicate using the MQTT protocol. The MQTT protocol has the advantages of being lightweight, having low bandwidth consumption, and supporting reconnection after disconnection, making it suitable for application scenarios where network conditions at bridge sites are unstable. Message transmission uses QoS level 1 to ensure that messages are delivered at least once, while critical warning messages use QoS level 2 to ensure that messages are delivered exactly once. Data transmission is encrypted using the TLS 1.3 protocol to prevent data from being stolen or tampered with during transmission. User authentication uses a certificate-based two-way authentication mechanism to ensure that only authorized devices can access the system.

[0089] The system's reliability design considers multiple failure modes, including hardware failures, network interruptions, and software anomalies. In one embodiment of this invention, the field data acquisition device is equipped with a local storage unit, which can temporarily store the most recent 7 days of acquired data during network interruptions and automatically re-upload it after the network is restored. The core processing server in the data center adopts a primary-backup redundancy configuration, automatically switching to the backup server when the primary server fails, with a switching time of less than 30 seconds. The database adopts a master-slave replication architecture to achieve real-time data backup. Critical business processes are configured with watchdog programs, automatically restarting when abnormal process exit is detected. The system automatically executes a self-check program every morning at midnight, checking the operating status of each module, storage space, network connectivity, and other indicators, and automatically sending alarm emails when anomalies are detected.

[0090] The system's scalability design supports flexible expansion of the monitoring scale. In one embodiment of the invention, the system architecture adopts a microservice design concept, with each functional module deployed and expanded independently. When the number of monitored bridges increases, the system's processing capacity can be improved by horizontally scaling processing nodes without requiring downtime for upgrades. Computationally intensive tasks such as image registration, defect detection, and joint width measurement support distributed parallel processing, and computing resources can be dynamically adjusted according to the load. Data storage adopts a sharding strategy, sharding data according to bridge number to avoid single-point storage bottlenecks. The system reserves standard interfaces for interfacing with third-party systems, supporting data integration with upper-level application systems such as bridge management information systems and traffic monitoring systems.

[0091] In terms of system deployment, this invention supports two modes: single-bridge deployment and centralized deployment across bridge clusters. In single-bridge deployment, the time-series image acquisition module is installed at the expansion joint of the bridge to be monitored, transmitting data to a local or cloud-based data center for processing via a wireless network. This mode is suitable for independent monitoring needs of important bridges, offering flexible deployment and convenient maintenance. In centralized deployment across bridge clusters, acquisition modules from multiple bridges share the processing resources of the same data center, monitored and scheduled through a unified management platform. This mode is suitable for large-scale monitoring scenarios involving highways or urban bridge clusters. In one embodiment of this invention, an expansion joint monitoring system covering 120 bridges was deployed in a provincial highway bridge cluster monitoring project, with a total of 280 monitoring points installed. All data was aggregated to the provincial transportation information center for centralized processing and display.

[0092] The system's human-computer interaction interface adopts a B / S architecture design, allowing users to access system functions through a standard web browser without installing dedicated client software. The interface design follows the principle of ease of use, providing intuitive data visualization and convenient operation entry points. In one embodiment of this invention, the system interface includes the following main functional modules: an overview panel displays the status distribution, early warning statistics, and key indicator trends of all monitored bridges; a bridge details page displays real-time images, historical data curves, and defect detection results for each expansion joint of a single bridge; an early warning management page provides functions for viewing, confirming, and recording the handling of early warning messages; a report generation page supports customized generation of maintenance recommendation reports based on time range and bridge scope; and a system configuration page provides configuration management functions for acquisition parameters, early warning thresholds, and user permissions.

[0093] The embodiments of the present invention are not limited to the specific embodiments described above. Those skilled in the art can make various equivalent changes or substitutions based on the technical solutions of the present invention, and all such changes or substitutions should be included within the protection scope of the present invention.

Claims

1. A method for monitoring bridge expansion joint defects using time-series image analysis, characterized in that, The method includes the following steps: S1. Time-series image acquisition steps: Install fixed industrial cameras on both sides of the bridge expansion joint, and periodically acquire high-resolution images of the expansion joint area according to the preset acquisition cycle. At the same time, acquire the ambient temperature data at the acquisition time to form a time-series image sequence containing timestamps, image data and temperature data. S2. Image registration processing steps: Feature points are extracted and matched from images acquired at different times in the time-series image sequence. Based on the matched feature point pairs, geometric transformation parameters between images are calculated. Images from different times are transformed to a unified reference coordinate system to achieve precise image alignment and eliminate interference caused by changes in ambient lighting and small camera displacements. S3. Defect detection and identification steps: Input the registered image into the pre-trained defect detection network to identify the types of defects in the expansion joint area. The defect types include rubber strip aging and cracking, steel loosening and warping, foreign object blockage in the gap, and concrete edge damage. Output the location information, boundary contour and confidence level of each type of defect. S4. Dynamic monitoring steps for seam width: The sub-pixel edge detection algorithm is used to extract the precise position of the edges on both sides of the expansion joint, and the opening and closing amount of the expansion joint is calculated as the seam width measurement value. Combined with the synchronously collected ambient temperature data, a seam width-temperature response curve model is established. Abnormal deformation behavior is identified by calculating the deviation between the measured seam width and the seam width predicted by the model. S5. Trend Analysis and Early Warning Steps: Compare historical time-series image sequences to analyze the evolution trend of the area, length, and severity of various defects over time, calculate the defect deterioration rate, mark the maintenance urgency level for expansion joints whose deterioration rate exceeds the early warning threshold, and generate a bridge expansion joint health status file and maintenance recommendation report.

2. The bridge expansion joint defect monitoring method based on time-series image analysis according to claim 1, characterized in that, In step S1, the fixed industrial camera has a resolution of no less than 20 million pixels, the acquisition period is from 1 hour to 7 days, and the acquisition accuracy of the ambient temperature data is no less than 0.5℃.

3. The bridge expansion joint defect monitoring method based on time-series image analysis according to claim 1, characterized in that, In step S2, the feature point extraction uses the SIFT algorithm or the ORB algorithm, the geometric transformation parameters are represented by the affine transformation matrix or the perspective transformation matrix, and the transformation matrix is ​​robustly estimated after removing mismatched points using the RANSAC algorithm.

4. The method for monitoring bridge expansion joint defects using time-series image analysis according to claim 1, characterized in that, In step S3, the disease detection network adopts a convolutional neural network structure based on an attention mechanism. It includes a feature extraction backbone network, a multi-scale feature fusion module, and a detection head module. The input image size of the disease detection network is 512×512 pixels to 1024×1024 pixels, and the output confidence threshold is set to 0.5 to 0.

8.

5. The bridge expansion joint defect monitoring method based on time-series image analysis according to claim 1, characterized in that, In step S4, the seam width-temperature response curve model is a linear model or a piecewise linear model. The linear model is expressed as the seam width equal to the product of the dynamic response coefficient and the temperature change plus the reference seam width. The dynamic response coefficient ranges from 0.5 mm / ℃ to 3.0 mm / ℃. The criterion for judging abnormal deformation behavior is that the measured seam width deviation exceeds three times the standard deviation of the predicted value of the response curve.

6. The method for monitoring bridge expansion joint defects using time-series image analysis according to claim 1, characterized in that, In step S4, the subpixel edge detection algorithm includes the following sub-steps: performing multi-scale Gaussian filtering preprocessing on the expansion joint edge region; calculating the horizontal and vertical gradients of the image using the Sobel operator; obtaining pixel-level edge positions by performing non-maximum suppression based on the gradient magnitude and gradient direction; and calculating subpixel-level edge coordinates in the neighborhood of the pixel-level edge position using the gradient-weighted centroid method, achieving a measurement accuracy of 0.1 to 0.02 pixels.

7. The bridge expansion joint defect monitoring method based on time-series image analysis according to claim 1, characterized in that, In step S5, the method for calculating the disease deterioration rate is as follows: extract the change values ​​of characteristic quantities of the same type of disease within two adjacent monitoring periods; divide the change values ​​of the characteristic quantities by the monitoring time interval to obtain the deterioration rate; the characteristic quantities include disease area, crack length, crack width, and disease severity index.

8. The method for monitoring bridge expansion joint defects using time-series image analysis according to claim 1, characterized in that, In step S5, the maintenance urgency level is divided into four levels: Level 1 is emergency maintenance, corresponding to a deterioration rate exceeding the first warning threshold or structural damage being detected; Level 2 is priority maintenance, corresponding to a deterioration rate between the second and first warning thresholds; Level 3 is planned maintenance, corresponding to a deterioration rate between the third and second warning thresholds; and Level 4 is observation and monitoring, corresponding to a deterioration rate below the third warning threshold.

9. The method for monitoring bridge expansion joint defects using time-series image analysis according to claim 1, characterized in that, In step S5, the bridge expansion joint health status file includes basic information of the expansion joint, historical image sequence, disease detection record, joint width monitoring data, temperature response curve parameters, deterioration trend curve and maintenance suggestions. The maintenance suggestion report includes disease type distribution statistics, deterioration risk assessment, maintenance priority ranking and maintenance measures suggestions.

10. A bridge expansion joint defect monitoring system based on time-series image analysis, used to implement the bridge expansion joint defect monitoring method based on time-series image analysis as described in any one of claims 1-9, characterized in that, The system includes: The time-series image acquisition module is used to install fixed industrial cameras on both sides of the bridge expansion joint, periodically acquire high-resolution images of the expansion joint area according to a preset acquisition cycle, and at the same time acquire ambient temperature data through a temperature sensor. The image registration and processing module is connected to the time-series image acquisition module and is used to extract and match feature points in images acquired at different times, calculate geometric transformation parameters, and transform the images to a unified reference coordinate system. The defect detection and identification module is connected to the image registration and processing module and is used to identify defects such as aging and cracking of rubber strips, loosening and warping of steel sections, blockage of gaps by foreign objects, and damage to concrete edges through a pre-trained defect detection network. The seam width dynamic monitoring module is connected to the image registration and processing module and the disease detection and identification module. It is used to measure the seam width using a sub-pixel edge detection algorithm, establish a seam width-temperature response curve, and identify abnormal deformation. The trend analysis and early warning module is connected to the disease detection and identification module and the crack width dynamic monitoring module. It is used to analyze the disease evolution trend, calculate the deterioration rate, mark the maintenance urgency level, and generate a health status file and maintenance recommendation report.

Images