Road and bridge structure health diagnosis method and system based on image analysis

Abstract

The invention provides a road and bridge structure health diagnosis method and system based on image analysis, and the method comprises the steps: firstly obtaining a surface image set of the same structure part of a target road and bridge structure at different monitoring time points and at different observation angles, and carrying out the time-space alignment processing to generate a time-space synchronous structure image group; performing feature extraction and correlation analysis on the structure image group through a pre-trained damage feature correlation model, generating a damage feature correlation set containing cross-view space damage features, cross-time point time damage features and correlation features thereof, and performing damage state conjoint analysis based on the damage feature correlation set. According to the method, the current damage state information and the damage development trend information are generated, finally, the road and bridge structure health diagnosis result is generated according to the current damage state information and the damage development trend information in combination with the preset health level judgment rule, potential problems of the road and bridge structure can be found in time, and safe operation of the road and bridge is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a method and system for diagnosing the health of road and bridge structures based on image analysis. Background Technology

[0002] In the field of road and bridge engineering, the health status of road and bridge structures is directly related to traffic safety and the safety of people's lives and property. Currently, health diagnosis of road and bridge structures mainly relies on traditional inspection methods, such as manual inspections, periodic load tests, and data collection using simple sensors. Manual inspections are inefficient and easily influenced by the subjective factors of the inspectors, making it difficult to detect subtle damage. Periodic load tests are costly and cannot reflect the real-time health status of road and bridge structures. Data collected using simple sensors is often limited, reflecting only certain aspects of the structure and failing to comprehensively and accurately assess its overall health. Furthermore, existing image analysis technologies have limitations in the health diagnosis of road and bridge structures. They typically only analyze images from a single perspective and at a single time point, failing to effectively integrate image information from different perspectives and time points, making it difficult to uncover hidden damage features and their correlations within the images, resulting in inaccurate and incomplete diagnoses of the health status of road and bridge structures. Summary of the Invention

[0003] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a method for health diagnosis of road and bridge structures based on image analysis, the method comprising:

[0004] Acquire a set of surface images of the target road and bridge structure, wherein the set of surface images includes surface images of the same structural part at different monitoring time points and different observation angles;

[0005] The surface image set is spatiotemporally aligned to generate a spatiotemporally synchronized structural image set;

[0006] The pre-trained damage feature association model is used to extract and analyze the structural image group to generate a damage feature association set. The damage feature association set includes cross-view spatial damage features, cross-time point temporal damage features, and the association features of the cross-view spatial damage features and cross-time point temporal damage features.

[0007] Based on the damage feature association set, perform joint damage status analysis to generate current damage status information and damage development trend information;

[0008] Based on the current damage status information and the damage development trend information, and combined with the preset health level determination rules, a road and bridge structure health diagnosis result is generated.

[0009] In another aspect, embodiments of the present invention also provide a road and bridge structure health diagnosis system based on image analysis, including a processor and a machine-readable storage medium connected to the processor. The machine-readable storage medium is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the machine-readable storage medium to implement the above-described method.

[0010] Based on the above, this invention acquires a set of surface images of the target road and bridge structure at different monitoring time points and observation angles, and performs spatiotemporal alignment processing to generate a spatiotemporally synchronized structural image set. This allows for a comprehensive and systematic acquisition of the appearance information of the road and bridge structure under different spatiotemporal conditions. A pre-trained damage feature association model is used to extract features and perform association analysis on the structural image set, generating a damage feature association set that includes cross-view spatial damage features, cross-time point temporal damage features, and their correlation features. This effectively uncovers hidden damage information and its inherent connections within the images, overcoming the limitations of traditional methods that can only analyze images from a single viewpoint or time point. Based on the damage feature association set, joint damage state analysis is performed, comprehensively considering damage information in both spatial and temporal dimensions to generate accurate current damage state information and damage development trend information. Finally, combined with preset health level judgment rules, a road and bridge structure health diagnosis result is generated, which helps to promptly identify potential problems in the road and bridge structure and ensure its safe operation. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the execution flow of the image analysis-based road and bridge structure health diagnosis method provided in an embodiment of the present invention.

[0012] Figure 2 This is a schematic diagram of exemplary hardware and software components of the image analysis-based road and bridge structure health diagnosis system provided in an embodiment of the present invention. Detailed Implementation

[0013] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating a method for diagnosing the health of road and bridge structures based on image analysis, provided in one embodiment of the present invention. The following is a detailed description of this method for diagnosing the health of road and bridge structures based on image analysis.

[0014] Step S110: Obtain a set of surface images of the target road and bridge structure, wherein the set of surface images includes surface images of the same structural part at different monitoring time points and different observation angles.

[0015] When conducting health diagnostics on a target road and bridge structure, the first step is to acquire a set of surface images. For example, for a highway bridge in use, to comprehensively understand the condition of specific pier structural parts, surface images need to be acquired at different times and from different angles. At different monitoring times, such as morning, noon, and evening, and on different dates, the bridge surface may exhibit different characteristics due to changes in environmental factors such as light and temperature. These changes help to identify potential damage. Simultaneously, acquiring images from different observation angles, such as the front, side, and top, avoids missing certain damage information due to a single perspective. By using image acquisition equipment, such as an industrial camera equipped with a high-precision lens and high-resolution sensor, images are captured from multiple angles at various monitoring times. In the morning, the pier is photographed from the front, recording its surface condition at that time and angle; at noon, the same pier is photographed again from the side, acquiring images from different perspectives. Through this continuous and multi-angle acquisition method, multiple sets of surface images of the same pier structural part at different times and angles are obtained, thus forming a surface image set for subsequent analysis.

[0016] Step S120: Perform spatiotemporal alignment processing on the surface image set to generate a spatiotemporally synchronized structural image group.

[0017] In this embodiment, since the images in the surface image set were acquired at different times and angles, there are temporal and spatial differences. To facilitate subsequent analysis, these images need to be spatiotemporally aligned. Images acquired at the same angle at different times may have positional deviations due to slight displacements of the bridge itself or minor changes in the camera position; images acquired at the same time from different angles will present different images due to different viewing angles.

[0018] To address these issues, time alignment is the first step. Images taken at different times from the same observation angle are sorted and calibrated chronologically to ensure accuracy. For bridge pier images taken on different mornings from the same frontal angle, they are arranged sequentially according to their acquisition time to guarantee temporal continuity in subsequent analyses.

[0019] Then, spatial alignment is performed using image registration technology. The image registration process mainly includes three steps: feature extraction, feature matching, and image transformation. In the feature extraction stage, representative feature points are identified from each image, such as corner points and edge points on the bridge pier surface. These feature points reflect key information in the image. In the feature matching stage, a matching algorithm based on feature descriptors is used to find the correspondence between feature points in different images. In the image transformation stage, based on the feature point matching results, transformation matrices between images are calculated, including rotation and translation matrices. By applying these transformation matrices to the images, they are spatially aligned, ensuring that the position and orientation of the same bridge pier structural component remain consistent across different images. After spatiotemporal alignment, a spatiotemporally synchronized set of structural images is generated.

[0020] Step S130: Perform feature extraction and association analysis on the structural image group using a pre-trained damage feature association model to generate a damage feature association set. The damage feature association set includes cross-view spatial damage features, cross-time point temporal damage features, and association features of the cross-view spatial damage features and cross-time point temporal damage features.

[0021] Step S131: Input the images from different perspectives at the same monitoring time point in the structural image group into the spatial feature extraction module of the damage feature association model. The spatial feature extraction module performs convolution operation on the images from each perspective through a convolutional network to extract the damage contour features from each perspective. The damage contour features include pixel-level features of crack boundaries, peeling area edges and deformation area edges.

[0022] After obtaining a set of spatiotemporally synchronized structural images, images from different perspectives at the same monitoring time point are input into the spatial feature extraction module of the damage feature association model. Taking a specific monitoring time as an example, images of bridge piers acquired from different perspectives such as the front, side, and top at that time are input into this module. The spatial feature extraction module uses a convolutional network to process these images. A convolutional network is a neural network specifically designed to process data with a grid structure, which can automatically extract feature information from images. During the convolution operation, the convolutional network uses multiple different convolutional kernels to slide across the image, performing convolution calculations on local regions of the image. Each convolutional kernel is multiplied element-wise with a local region of the image and summed to obtain a convolution result.

[0023] By continuously moving the convolution kernel, a series of convolution results can be obtained at different locations in the image, and these convolution results constitute a feature map. Different convolution kernels can extract different types of features, such as edge features and texture features. In this embodiment, the focus is on extracting damage contour features from various viewpoints, including pixel-level features of crack boundaries, peeling area edges, and deformation area outer edges. Crack boundaries are the edge positions of cracks in the image, which can clearly define the extent of the crack; peeling area edges are the edges of the areas where the surface material of the road and bridge structure has peeled off; and deformation area outer edges are the outer boundaries of the areas where the structure has deformed.

[0024] During the convolution operation, the convolutional network continuously learns and adjusts the parameters of the convolution kernel, making the extracted damage contour features more accurate and clear. Through multiple convolution and pooling operations, the size of the feature map is gradually reduced, while the expressive power of the features is enhanced. Ultimately, the spatial feature extraction module can extract rich damage contour features from images from various viewpoints.

[0025] Step S132: Perform cross-view consistency verification on the damage contour features under each viewpoint. By comparing the overlapping areas of the features, filter out the damage contours that appear repeatedly in at least two viewpoints and generate cross-view spatial damage features. The cross-view spatial damage features include information on the stable damage location, shape and coverage.

[0026] Step S1321: Convert the damage contour features from each viewpoint into a binary feature map, where the pixel value of the damaged area is 1 and the pixel value of the non-damaged area is 0.

[0027] After obtaining the damage contour features from various viewpoints, these features need to be converted into binarized feature maps to facilitate cross-viewpoint consistency verification. Binarization is an image processing method that simplifies pixel values ​​in an image into two states. For each viewpoint's damage contour features, a suitable threshold is set, assigning pixel values ​​of damaged areas to 1 and non-damaged areas to 0. Taking the bridge pier damage contour features from a certain viewpoint as an example, pixels in damaged areas are uniformly assigned a value of 1, while pixels in non-damaged areas are assigned a value of 0. This converts the damage contour features from that viewpoint into a binarized feature map. By performing the same processing on the damage contour features from all viewpoints, a set of binarized feature maps is obtained. These binarized feature maps can more clearly show the distribution of damaged and non-damaged areas.

[0028] Step S1322: The binarized feature map is superimposed at the pixel level to obtain a superimposed feature map, wherein the value of each pixel in the superimposed feature map is the number of viewpoints covering that pixel.

[0029] After obtaining the binarized feature maps, they can be pixel-level superimposed. Pixel-level superimposition involves adding the pixel values ​​at the same location from multiple binarized feature maps. For each pixel location, the pixel values ​​at that location from all viewpoints are summed to obtain the value of that pixel in the superimposed feature map. Taking a pixel on the surface of a bridge pier as an example, if this pixel belongs to the damaged area (pixel value 1) in the binarized feature maps from different viewpoints, then the value of this pixel in the superimposed feature map will be 3, indicating that damage is detected at that location in all three viewpoints. By performing the above superimposition operation on all pixels, the superimposed feature map is obtained. The superimposed feature map can reflect which areas are detected as damaged in multiple viewpoints.

[0030] Step S1323: Extract the pixel regions with values ​​greater than or equal to 2 from the superimposed feature map as the damage regions consistent across viewpoints.

[0031] After obtaining the overlay feature map, damage regions consistent across viewing angles are extracted based on pixel values. Since a value greater than or equal to 2 indicates that the pixel region is detected as having damage in at least two viewing angles, pixel regions with values ​​greater than or equal to 2 in the overlay feature map are extracted as damage regions consistent across viewing angles. Taking pixels in the overlay feature map as an example, for pixels with values ​​of 2 or greater, their regions are marked; these regions are those where damage can be determined from multiple viewing angles. This method allows for the selection of stably existing damage regions, eliminating false positives that may be caused by errors in a single viewing angle and improving the accuracy of damage detection.

[0032] Step S1324: Perform morphological closing operation on the cross-view consistent damage area to fill the small holes in the damage area and smooth the edges to obtain a continuous damage contour.

[0033] After extracting the damage region consistent across different viewpoints, morphological closing operations are performed on this region to obtain a more accurate and continuous damage contour. Morphological closing is an image processing method based on morphological operations, consisting of two operations: dilation and erosion. First, dilation is performed on the damage region consistent across different viewpoints, expanding it outwards and filling in small holes and cracks. Then, erosion is performed, shrinking the damage region inwards and removing unnecessary edge noise. This dilation-erosion closing operation fills in small holes within the damage region and smooths its edges, resulting in a more continuous and clear damage contour. Taking the damage region consistent across different viewpoints on the bridge pier surface as an example, after morphological closing, the damage region, which originally contained small holes and discontinuous edges, becomes more complete and smoother, resulting in a continuous damage contour.

[0034] Step S1325: Calculate the minimum bounding rectangle of the damage profile, and record the boundary vertex coordinates, width, and height of the minimum bounding rectangle as a quantitative description of the damage location and coverage area.

[0035] After obtaining a continuous damage profile, the minimum bounding rectangle of the damage profile is calculated to quantify the location and coverage of the damage. The minimum bounding rectangle is the smallest rectangle that can completely contain the damage profile. Using conventional boundary point search algorithms in related technologies, the boundary points of the damage profile are found, and then the minimum rectangle that can enclose these boundary points is determined. The coordinates of the boundary vertices of this minimum bounding rectangle are recorded; these coordinates accurately locate the damage location. Simultaneously, the width and height of the minimum bounding rectangle are recorded; these two parameters reflect the size of the damage coverage. Taking the damage profile of a bridge pier surface as an example, after calculating its minimum bounding rectangle, the coordinates of its four vertices, as well as its width and height, are recorded. This constitutes a quantitative description of the damage location and coverage, allowing the damage situation to be expressed in a concrete way.

[0036] Step S1326: Organize the shape information of the damage contour and the quantization description into the cross-view spatial damage feature.

[0037] After obtaining the shape information and quantitative description of the damage contour, these are organized into cross-view spatial damage features. The shape information of the damage contour can include different morphological identifiers such as linear, sheet-like, or irregular shapes, which can reflect the type and characteristics of the damage. The quantitative description includes information such as the coordinates of the boundary vertices, width, and height of the minimum bounding rectangle of the damage contour, which can accurately describe the location and coverage of the damage. Integrating the above shape information and quantitative description forms the cross-view spatial damage features. Taking the damage on the surface of a bridge pier as an example, the linear shape information of its damage contour and the relevant quantitative description of the minimum bounding rectangle are combined to obtain the cross-view spatial damage features of the damage. These cross-view spatial damage features contain information on the stable location, shape, and coverage of the damage.

[0038] Step S133: Input the images of the same viewpoint at different time points in the structural image group into the time feature extraction module of the damage feature association model. The time feature extraction module calculates the pixel difference of the damage contour at adjacent time points through continuous frame difference operation, extracts the boundary movement trajectory of the damage expansion area, and generates cross-time point time damage features. The cross-time point time damage features include time series information of damage length growth path, width expansion direction and area increase rate.

[0039] Step S1331: Perform pixel-level difference on the damage contour feature maps of adjacent time points under the same viewpoint to obtain a difference feature map. The area with a pixel value of 1 in the difference feature map represents newly added damage or disappeared damage.

[0040] After obtaining the structural image set, images from different time points under the same viewpoint are input into the temporal feature extraction module of the damage feature association model. Taking a certain viewpoint of the bridge pier as an example, the damage contour feature maps corresponding to the images of that viewpoint acquired at different time points are processed. Pixel-level differencing is performed on the damage contour feature maps of adjacent time points. Pixel-level differencing involves subtracting the pixel values ​​at the same position in the damage contour feature maps of adjacent time points. If the two pixel values ​​at adjacent time points are different, the pixel value at that position in the differencing feature map is 1, indicating that new damage has occurred or existing damage has disappeared; if the pixel values ​​are the same, the pixel value at that position in the differencing feature map is 0, indicating that the damage situation in that area has not changed. By performing the above pixel-level differencing operation on the damage contour feature maps of adjacent time points under the same viewpoint, a differencing feature map is obtained. The differencing feature map can clearly show the changes in damage between adjacent time points.

[0041] Step S1332: Retain the region with a pixel value of 1 in the differential feature map that is located outside the original damage contour as the newly added damage extension region.

[0042] After obtaining the differential feature map, to determine the extent of damage expansion, it is necessary to filter out newly added damage expansion regions from the differential feature map. Only areas in the differential feature map with a pixel value of 1 that are located outside the original damage contour are retained as newly added damage expansion regions. This is because a pixel value of 1 indicates that damage has occurred in that area, and being located outside the original damage contour indicates that this is a newly emerging damage expansion. Taking bridge pier damage as an example, in the differential feature map, areas with a pixel value of 1 that are outside the original damage contour are marked; these areas are the newly added damage expansion regions. Through the above filtering operation, the location of damage expansion can be accurately located.

[0043] Step S1333: Extract the coordinates of the boundary points of the newly added damage expansion area, and record the displacement direction and distance of each boundary point relative to the damage contour boundary at the previous time point.

[0044] After identifying the newly added damage expansion area, to further analyze the damage expansion, it is necessary to extract the coordinates of the boundary points of the newly added damage expansion area and record the displacement direction and distance of each boundary point relative to the damage contour boundary at the previous time point. Using conventional boundary localization algorithms in related technologies, the boundary points of the newly added damage expansion area are identified, and the coordinates of these boundary points are recorded. Then, the coordinates of these boundary points are compared with the coordinates of the corresponding points on the damage contour boundary at the previous time point to calculate the displacement direction and distance of each boundary point relative to the damage contour boundary at the previous time point. Taking the newly added damage expansion area of ​​a bridge pier as an example, for each boundary point in this area, the direction of its movement relative to the damage contour boundary at the previous time point—whether left, right, upward, or downward, etc.—is determined, as well as the magnitude of the movement. By recording the above displacement direction and distance, the trend of damage expansion can be intuitively understood.

[0045] Step S1334: Statistically analyze the displacement data of the boundary points, calculate the average displacement direction and average displacement distance, and use them as the main direction and average rate of damage propagation.

[0046] After recording the displacement direction and distance of each boundary point relative to the damage contour boundary at the previous time point, statistical analysis was performed on these displacement data. The average displacement direction, representing the main direction of damage propagation, was obtained by summing and averaging the displacement directions of all boundary points. The average displacement distance, representing the average rate of damage propagation, was obtained by summing and averaging the displacement distances of all boundary points. Taking the damage propagation of a bridge pier as an example, statistical calculations of the displacement directions and distances of all boundary points revealed that the main direction of damage propagation is towards a specific direction, and the average rate is a relatively stable value. Through the above statistical analysis, the overall damage propagation situation can be grasped.

[0047] Step S1335: Track the coordinate changes of the set boundary point of the same damage contour at different time points, and generate the movement trajectory curve of the set boundary point.

[0048] To gain a more detailed understanding of the damage propagation process, the coordinate changes of defined boundary points on the same damage profile at different time points are tracked. One or more defined boundary points on the damage profile are selected, and their coordinates are recorded at different time points. Over time, these coordinates are sequentially connected to generate the movement trajectory curve of the defined boundary point. Taking the damage profile of a bridge pier as an example, a specific boundary point on the damage profile is selected, and its coordinates are collected at different time points. These coordinates are then connected in a coordinate system to form a movement trajectory curve. This movement trajectory curve can visually demonstrate the positional changes of the damage boundary point at different times, reflecting the dynamic process of damage propagation.

[0049] Step S1336: Organize the average displacement direction, average displacement distance, and movement trajectory curve into the time damage characteristics across time points.

[0050] After obtaining the average displacement direction, average displacement distance, and movement trajectory curve, these are organized into a cross-time point temporal damage feature. The average displacement direction reflects the main direction of damage propagation, the average displacement distance represents the average rate of damage propagation, and the movement trajectory curve shows the positional changes of the damage boundary point at different times. These three parameters are integrated to form the cross-time point temporal damage feature. Taking bridge pier damage as an example, the average displacement direction, average displacement distance, and movement trajectory curve of the set boundary point are combined to obtain the cross-time point temporal damage feature of this damage. This cross-time point temporal damage feature includes time-series information on the damage length growth path, width expansion direction, and area increase rate.

[0051] Step S134: Input the cross-view spatial damage features and the cross-time point temporal damage features into the association analysis module of the damage feature association model. The association analysis module establishes the correspondence between the spatial damage location and the temporal expansion trajectory through feature location matching, and generates a damage feature association set. The association set contains the association rules of the damage location's expansion path, expansion rate and initial damage scale over time.

[0052] For example, step S1341: extract the shape information and quantization description of the damage contour from the cross-view spatial damage features. The shape information includes morphological identifiers of linear, sheet-like or irregular shapes, and the quantization description includes the boundary vertex coordinates, width and height of the minimum bounding rectangle of the damage contour.

[0053] After obtaining the cross-view spatial damage features and cross-time point temporal damage features, these features are input into the correlation analysis module of the damage feature correlation model. First, the shape information and quantitative description of the damage contour are extracted from the cross-view spatial damage features. The shape information of the damage contour reflects the type and characteristics of the damage, including different morphological identifiers such as linear, sheet-like, or irregular shapes. The quantitative description includes information such as the coordinates of the boundary vertices, width, and height of the minimum bounding rectangle of the damage contour. This information can accurately locate the damage and describe the size of the damage coverage area. Taking the cross-view spatial damage features of the bridge pier surface as an example, the morphological identifier of the linear shape of the damage contour, as well as the coordinates of the boundary vertices, width, and height of its minimum bounding rectangle, are extracted.

[0054] Step S1342: Extract the average displacement direction, average displacement distance, and movement trajectory curve from the cross-time point time damage features. The movement trajectory curve contains the coordinate sequence of the damage boundary point at different time points.

[0055] In this embodiment, relevant information is extracted from the time-series damage characteristics. The average displacement direction reflects the main trend of damage propagation, the average displacement distance reflects the average speed of damage propagation, and the movement trajectory curve records the specific positional changes of the damage boundary point at different time points. The movement trajectory curve is composed of the coordinate sequence of the damage boundary point at different time points, and these coordinate sequences can intuitively show the dynamic process of damage propagation. Taking the damage to the bridge pier as an example, the movement trajectory curve composed of the average displacement direction, average displacement distance, and coordinates of the damage boundary point at different time points is extracted.

[0056] Step S1343: Match the coordinates of the boundary vertex of the minimum bounding rectangle in the quantization description with the coordinates of the initial time point of the movement trajectory curve to determine the positional correspondence between the starting point of the movement trajectory curve and the spatial damage contour.

[0057] After obtaining the quantitative description of spatial damage features across different viewpoints and the movement trajectory curves of temporal damage features across different time points, a position matching operation is performed. The coordinates of the boundary vertices of the minimum bounding rectangle of the damage contour in the quantitative description are compared with the coordinates of the initial time point of the movement trajectory curve. Using conventional matching algorithms, the positional correspondence between the starting point of the movement trajectory curve and the spatial damage contour is determined. For damage on the pier surface, the coordinates of the boundary vertices of the minimum bounding rectangle and the coordinates of the initial time point of the movement trajectory curve are precisely matched to determine the specific location of the starting point of the movement trajectory curve within the spatial damage contour. This positional correspondence links the spatial location of the damage to the temporal starting point of the damage expansion.

[0058] Step S1344: Analyze the correlation between the extension direction of the moving trajectory curve and the shape of the damage contour in the shape information. If the damage contour is linear, determine whether the moving trajectory extends along the long axis of the linear contour; if it is sheet-like, determine whether the moving trajectory extends outward towards the edge of the sheet-like contour.

[0059] After location matching is completed, the correlation between the extension direction of the movement trajectory curve and the morphology of the damage contour is further analyzed. Based on different damage contour morphologies, such as linear, sheet-like, or irregular shapes, it is determined whether the extension of the movement trajectory conforms to the corresponding pattern. If the damage contour is linear, it is observed whether the movement trajectory extends along the long axis of the linear contour. Because linear damage usually expands further along its long axis, this judgment helps predict the direction of damage development. If the damage contour is sheet-like, it is determined whether the movement trajectory extends outwards towards the edge of the sheet-like contour; sheet-like damage generally expands gradually at the edge. Taking linear damage on the surface of a bridge pier as an example, analyzing whether its movement trajectory curve develops along the long axis of the linear contour, through the above correlation analysis, allows for a deeper understanding of the damage's expansion characteristics.

[0060] Step S1345: Calculate the ratio between the average displacement distance and the width and height of the damage profile in the quantification description, and generate the correlation parameters between the damage propagation rate and the spatial damage size.

[0061] To gain a more comprehensive understanding of damage propagation, the proportional relationship between the average displacement distance and the width and height of the damage profile is calculated. The average displacement distance represents the average rate of damage propagation, while the width and height of the damage profile reflect the spatial dimensions of the damage. Therefore, by calculating the proportional relationship between them, a correlation parameter between the damage propagation rate and the spatial damage size can be obtained. This correlation parameter reflects the intrinsic link between the damage propagation rate and the size of the damage itself. For damage on the pier surface, the ratio of its average displacement distance to the width and height of the damage profile is calculated, thereby generating a correlation parameter reflecting the relationship between the damage propagation rate and the spatial damage size. This correlation parameter helps in analyzing the interaction between the rate of damage propagation and the scale of the damage.

[0062] Step S1346: Track the coordinate changes of subsequent time points in the movement trajectory curve to verify whether it continues to conform to the extension pattern corresponding to the shape information.

[0063] After establishing the correlation between the movement trajectory curve and the damage profile morphology, and obtaining the correlation parameters between the damage propagation rate and the spatial damage size, it is necessary to track and verify the subsequent development of the movement trajectory curve. Track the coordinate changes of subsequent time points in the movement trajectory curve and check whether it continues to conform to the extension pattern determined based on the damage profile morphology. If the damage profile is linear, verify whether the subsequent movement trajectory still extends along the long axis of the linear profile; if it is sheet-like damage, check whether the movement trajectory continues to develop towards the outward expansion direction of the sheet-like profile edge. Taking the damage on the bridge pier surface as an example, continuously track the coordinates of subsequent time points in the movement trajectory curve and observe whether it always follows the extension pattern of linear damage. Through the above verification process, the accuracy and reliability of the established spatiotemporal correlation can be ensured, further improving the accuracy of damage development trend prediction.

[0064] Step S1347: Record the position correspondence, extension direction correlation, extension rate and size correlation parameters and extension law verification results to generate the spatiotemporal correlation features of damage extension.

[0065] The location correspondence, extension direction correlation, expansion rate and size correlation parameters, and extension law verification results obtained in the above steps are recorded. These information collectively constitute the spatiotemporal correlation characteristics of damage expansion. The location correspondence determines the initial connection between the spatial damage location and the temporal expansion trajectory; the extension direction correlation reflects the relationship between the damage expansion direction and the damage contour morphology; the expansion rate and size correlation parameters embody the intrinsic connection between the damage expansion rate and the spatial size of the damage; and the extension law verification results ensure the reliability of the above correlations. Taking bridge pier damage as an example, the above information is organized and recorded to generate the spatiotemporal correlation characteristics of the damage expansion. These spatiotemporal correlation characteristics can comprehensively describe the development characteristics of the damage in space and time.

[0066] Step S1348: Bind the spatiotemporal correlation features with the shape information, quantitative description, average displacement direction, average displacement distance, and movement trajectory curve of the cross-view spatial damage features, and generate the damage feature association set. The damage feature association set includes the positional correspondence between spatial damage location and temporal expansion trajectory, the correlation law between expansion direction and damage morphology, and the correlation parameters between expansion rate and damage size.

[0067] Finally, the spatiotemporal correlation features are bound to the shape information and quantitative description of cross-view spatial damage features, as well as the average displacement direction, average displacement distance, and movement trajectory curve of cross-time point temporal damage features, forming a damage feature association set. This damage feature association set integrates relevant information of spatial and temporal damage, including important content such as the positional correspondence between spatial damage location and temporal expansion trajectory, the correlation between expansion direction and damage morphology, and the correlation parameters between expansion rate and damage size. By binding the above information, the characteristics and development laws of damage can be described more comprehensively and systematically. Taking the damage on the surface of a bridge pier as an example, the spatiotemporal correlation features are integrated with the relevant information of cross-view spatial damage features and cross-time point temporal damage features to generate a complete damage feature association set.

[0068] Step S135: Perform feature enhancement processing on the cross-view spatial damage features, the cross-time point temporal damage features, and the associated features. Enhance the pixel response of the damage boundary by adjusting the local contrast, suppress background interference in the non-damaged area, and finally output the damage feature association set.

[0069] After obtaining cross-view spatial damage features, cross-time point temporal damage features, and associated features, feature enhancement processing is performed to improve the quality and recognizability of these features. Feature enhancement is mainly achieved through local contrast adjustment. Local contrast adjustment can strengthen the pixel response of damage boundaries, making the damage boundaries clearer and more obvious in the image, while suppressing background interference in non-damaged areas and reducing the impact of unnecessary noise and interference information on feature analysis. Taking the damage features of the bridge pier surface as an example, local contrast adjustment is performed on the cross-view spatial damage features, cross-time point temporal damage features, and associated features. Enhancing the pixel response of damage boundaries makes the damage outline more prominent, facilitating subsequent analysis and recognition; suppressing background interference in non-damaged areas makes the damage features purer and more accurate. After feature enhancement processing, the final output is an optimized damage feature association set, in which the features are clearer and more accurate.

[0070] Step S140: Perform joint damage status analysis based on the damage feature association set to generate current damage status information and damage development trend information.

[0071] Step S141: Extract cross-view spatial damage features from the damage feature association set, separate the damaged area from the non-damaged area using an image segmentation algorithm, calculate the maximum length, maximum width and total area of ​​the damaged area, and generate damage area proportion data by combining the standard image size of the corresponding structural part.

[0072] After obtaining the damage feature association set, cross-view spatial damage features are first extracted from it. To accurately assess the current damage state, damaged and non-damaged regions need to be separated. Image segmentation algorithms are used to achieve this goal, as they can distinguish between damaged and non-damaged regions based on pixel features and texture information. Taking the cross-view spatial damage features of a bridge pier surface as an example, image segmentation algorithms are used to segment the damaged region from the entire image. After separating the damaged region, the maximum length, maximum width, and total area of ​​the damaged region are calculated. The maximum length and maximum width describe the extent of damage in different directions, while the total area reflects the overall scale of the damage. By combining the standard image size of the corresponding structural part, the area of ​​the damaged region is compared with the area of ​​the standard image to generate damage region proportion data. This damage region proportion data can intuitively reflect the proportion of damage in the entire structural part.

[0073] Step S142: Extract time-series damage features from the damage feature association set, calculate the historical average value of damage propagation rate through time series analysis, and generate trend data of damage propagation rate by combining the current propagation rate.

[0074] Next, time-series damage features across time points are extracted from the damage feature association set. To analyze the changes in the damage propagation rate, time series analysis is employed. Time series analysis processes and analyzes damage propagation rate data at different time points, calculating the historical average damage propagation rate. The historical average reflects the average propagation speed of damage over a past period. By combining the current propagation rate with the historical average, trend data on the damage propagation rate is generated. This trend data can determine whether the damage propagation rate is accelerating, slowing down, or remaining stable. Taking bridge pier damage as an example, the historical average damage propagation rate is calculated through time series analysis and then compared with the current propagation rate to generate trend data on the damage propagation rate. By analyzing this trend data, the future rate of damage development can be predicted.

[0075] Step S143: Perform correlation analysis on the data of the proportion of the damaged area and the data of the trend of the damage expansion rate to determine whether the damage expansion rate accelerates as the damaged area increases, and screen out the key damaged areas that expand rapidly.

[0076] After obtaining the data on the proportion of damaged areas and the trend data on the rate of damage expansion, a correlation analysis is performed on them. Correlation analysis can determine whether there is an intrinsic relationship between the rate of damage expansion and the size of the damaged area, i.e., whether the rate of damage expansion accelerates as the damaged area increases. A two-dimensional data set of damaged area proportion and expansion rate is established, containing the damaged area proportion values ​​and corresponding expansion rate values ​​at different time points. The correlation coefficient of this two-dimensional data set is calculated to measure the degree of linear correlation between the damaged area proportion and the expansion rate. If the correlation coefficient is greater than a preset threshold, it is determined that the rate of damage expansion accelerates as the damaged area increases; if the correlation coefficient is less than or equal to the preset threshold, it is determined that there is no significant correlation between the rate of damage expansion and the size of the damaged area. Combining the determination results with the location information of the damaged areas, damaged areas that simultaneously meet the criteria of large area proportion, accelerated expansion rate, and location near critical structural components are selected as key damaged areas requiring focused attention. Taking bridge pier damage as an example, a correlation analysis is performed on the data on the proportion of damaged areas and the trend data on the rate of damage expansion, and key damaged areas meeting the criteria are selected based on the analysis results. Identifying key damaged areas helps to concentrate resources on targeted monitoring and maintenance, improving the efficiency and effectiveness of damage management.

[0077] Step S144: Generate current damage status information based on the location, damage type, current area proportion, and expansion rate of the key damaged area. The current damage status information includes damage location coordinates, type identifier, and severity description.

[0078] After identifying key damaged areas, current damage status information is generated based on relevant information from these areas. The location of key damaged areas can be precisely represented by coordinates, and the damage type is identified based on damage characteristics, such as cracks, peeling, and deformation. The current area proportion reflects the percentage of damage within the structural component, and the expansion rate reflects the speed of damage development. Combining this information, current damage status information is generated, including damage location coordinates, type identification, and severity description. For key damaged areas on the pier surface, their location coordinates are recorded, the damage type is determined to be cracks, and the severity of the damage is described by combining the damage area proportion and expansion rate, such as mild, moderate, or severe. The generated current damage status information accurately reflects the actual damage situation.

[0079] Step S145: Using the damage propagation path prediction algorithm, based on the propagation trajectory information in the damage feature association set, simulate the damage profile at future monitoring time points, predict the new areas that the damage may cover and the direction of change in the propagation rate, and generate damage development trend information. The damage development trend information includes the future location range of the damage, the predicted value of the propagation rate, and a list of key structural components that may be affected.

[0080] To predict the future development of damage, a damage propagation path prediction algorithm is employed. This algorithm is based on propagation trajectory information from a damage feature association set, which records the damage's development over time. By analyzing and modeling this propagation trajectory information, the damage profile at future monitoring points is simulated. The algorithm predicts new areas that the damage may cover, i.e., it infers the areas that the damage may extend to in the future based on the damage propagation trend. Simultaneously, by combining the historical patterns of propagation rate changes, the algorithm predicts the increment or decrease in the future damage propagation rate, obtaining a predicted propagation rate value. Furthermore, based on the predicted damage location, it determines whether the damage will extend to critical structural components, generating a list of potentially affected critical structural components. For bridge pier damage, the damage propagation path prediction algorithm simulates the future damage profile, predicts new areas that the damage may cover and the direction of change in the propagation rate, and identifies potentially affected critical structural components, such as the support connection area and the mid-span area of ​​the beam. The generated damage development trend information includes the future location range of the damage, the predicted propagation rate value, and a list of potentially affected critical structural components.

[0081] Step S1451: Extract historical data of damage propagation trajectory from the damage feature association set, wherein the historical data includes the coordinate sequence of damage boundary points at different time points.

[0082] When using the damage propagation path prediction algorithm, historical data on damage propagation trajectories are first extracted from the damage feature association set. This historical data consists of coordinate sequences of damage boundary points at different time points, recording the specific development process of the damage over time. Taking damage on the surface of a bridge pier as an example, the coordinates of damage boundary points at multiple time points are extracted from the damage feature association set. These coordinates are arranged in chronological order to form historical data on the damage propagation trajectory.

[0083] Step S1452: Use a polynomial fitting algorithm to fit the coordinate sequence to generate the motion equation of the damage boundary point. The motion equation describes the change of the boundary point position over time.

[0084] After extracting historical data on damage propagation trajectories, a polynomial fitting algorithm is used to fit these coordinate sequences. The polynomial fitting algorithm finds a suitable polynomial function based on given coordinate points, making the function approximate these coordinate points as closely as possible. Through polynomial fitting, the motion equations of the damage boundary points are generated. These motion equations describe the changes in the position of the damage boundary points over time and reflect the trend of damage propagation. For the coordinate sequence of bridge pier damage boundary points, a suitable polynomial function is found using the polynomial fitting algorithm. This polynomial function accurately describes the positional changes of the damage boundary points at different times. The generated motion equations provide a mathematical model for subsequent prediction of future damage profiles.

[0085] Step S1453: Extrapolate the boundary point coordinates of future monitoring time points based on the motion equation to generate the predicted boundary of the future damage profile.

[0086] After obtaining the equations of motion for the damage boundary points, the coordinates of the boundary points at future monitoring time points are extrapolated from these equations. By substituting the future monitoring time points into the equations of motion, the corresponding boundary point coordinates are calculated. Connecting these boundary point coordinates at future monitoring time points generates a predicted boundary for the future damage profile. Taking bridge pier damage as an example, the coordinates of the damage boundary points at a future monitoring time point are calculated based on the equations of motion, and these coordinates are connected sequentially to form the predicted boundary for the future damage profile. This predicted boundary can visually demonstrate the possible future extent of the damage.

[0087] Step S1454: Calculate the area difference between the predicted boundary and the current damage profile, and predict the increment or decrease of the future damage expansion rate by combining the historical expansion rate variation pattern.

[0088] After generating the predicted boundary of the future damage profile, the area difference between the predicted boundary and the current damage profile is calculated. This area difference reflects the potential area of ​​future damage expansion. Combining historical patterns of expansion rate changes, the method of analyzing how the damage expansion rate changes with time and damage area is analyzed. Based on these patterns, the increment or decrease in the future damage expansion rate is predicted. If the historical damage expansion rate has accelerated with increasing damage area, then based on the predicted area difference, the predicted future damage expansion rate may have a certain increment; conversely, if the historical expansion rate has been relatively stable or has slowed down with increasing area, then the predicted future expansion rate may remain stable or have a certain decrease. For bridge pier damage, the area difference between the predicted boundary and the current damage profile is calculated, and combined with historical patterns of expansion rate changes, the change in the future damage expansion rate is predicted.

[0089] Step S1455: Based on the location information of the predicted boundary, determine whether the damage extends to the critical structural components, and generate a list of critical structural components that may be affected. The critical structural components include the support connection area and the mid-span area of ​​the beam.

[0090] Based on the location information of the predicted boundary of the future damage profile, it is determined whether the damage will extend to critical structural components. Critical structural components play a vital role in the safety and stability of the entire road and bridge structure, such as the bearing connection area and the mid-span area of ​​the beam. If the predicted boundary is close to or covers these critical structural components, then it is determined that the damage may affect them. Potentially affected critical structural components are recorded to generate a list of potentially affected critical structural components. Taking pier damage as an example, based on the location information of the predicted boundary, it is determined whether the damage will extend to critical structural components such as the bearing connection area and the mid-span area of ​​the beam. If the predicted boundary is close to or covers these critical structural components, they are included in the list of potentially affected critical structural components. This list of critical structural components helps staff understand the potentially serious consequences of damage in advance, thereby enabling targeted protection and maintenance measures.

[0091] Step S1456: Organize the predicted boundary coordinates, predicted propagation rate values, and list of key structural components into the damage development trend information.

[0092] After completing the above calculations and judgments, the predicted boundary coordinates, predicted propagation rates, and a list of potentially affected key structural components are compiled to form damage development trend information. The predicted boundary coordinates clarify the potential area covered by future damage, the predicted propagation rate reflects the future speed of damage development, and the list of key structural components indicates the important areas that may be affected by the damage. Integrating this information provides a comprehensive reflection of the future development trend of the damage. For bridge pier damage, compiling the predicted boundary coordinates, predicted propagation rates, and a list of potentially affected key structural components such as the bearing connection area and the mid-span area of ​​the beam forms complete damage development trend information, which helps in advance planning and taking corresponding measures to address the development of damage.

[0093] Step S1411: Perform grayscale threshold segmentation on the image region corresponding to the cross-view spatial damage feature, and mark the region with pixel grayscale value greater than the damage threshold as the damaged region, and the rest as the non-damaged region.

[0094] After extracting cross-view spatial damage features from the damage feature association set, gray-level thresholding is performed on the image regions corresponding to the cross-view spatial damage features to separate damaged and non-damaged regions. Gray-level thresholding is a segmentation method based on image gray values. By setting a damage threshold, pixels in the image are divided into damaged and non-damaged regions. For the image corresponding to the cross-view spatial damage features of the bridge pier surface, regions with pixel gray values ​​greater than the damage threshold are marked as damaged regions because the pixel features in these regions may be related to damage; while regions with pixel gray values ​​less than or equal to the damage threshold are marked as non-damaged regions. This method initially distinguishes between damaged and non-damaged regions.

[0095] Step S1412: Use a connected component analysis algorithm to identify all connected components in the damaged region, where each connected component represents an independent damaged region.

[0096] After grayscale thresholding, connected component analysis (CBI) is used to further process the damaged region. CBI identifies all connected components within the damaged region, with each component representing an independent damaged area. In the damaged area on the bridge pier surface, there may be multiple independent damaged parts; CBI can identify these independent damaged parts. The algorithm divides adjacent pixels with similar characteristics into the same connected component based on the connectivity between pixels, thus subdividing the damaged region into multiple independent damaged areas, facilitating subsequent individual analysis and calculation of each damaged area.

[0097] Step S1413: For each connected component, calculate the minimum bounding rectangle of its contour, where the length of the long side of the minimum bounding rectangle is the maximum length of the damaged region and the length of the short side is the maximum width of the damaged region.

[0098] After identifying all connected components within the damaged region, further parameter calculations are performed on each component. The minimum bounding rectangle of the contour of each connected component is calculated; this minimum bounding rectangle is the smallest rectangle that completely encloses the connected component. The length of the longer side of the minimum bounding rectangle represents the maximum extent of the damaged region in the longitudinal direction, i.e., the maximum length of the damaged region; the length of the shorter side represents the maximum extent of the damaged region in the shorter direction, i.e., the maximum width of the damaged region. For each independent damaged region (connected component) on the pier surface, its minimum bounding rectangle is calculated, yielding the maximum length and maximum width of each damaged region. These parameters more accurately describe the size and shape of the damaged region.

[0099] Step S1414: Count the number of pixels in each connected component, and convert the number of pixels into the actual area based on the resolution of the image acquisition device, which is the total area of ​​the damaged region.

[0100] After obtaining the maximum length and maximum width of each connected component, the number of pixels in each connected component is counted. The number of pixels reflects the size of the damaged area in the image, but to obtain the actual area, it needs to be converted according to the resolution of the image acquisition device. The resolution of the image acquisition device determines the actual physical size represented by each pixel. By multiplying the number of pixels by the actual area represented by each pixel, the number of pixels can be converted into the actual area. For each independent damaged area (connected component) on the pier surface, its number of pixels is counted, and the number of pixels is converted into the actual area according to the resolution of the image acquisition device. The actual areas of all connected components are added together to obtain the total area of ​​the damaged area. This total area can accurately reflect the size of the damage in the actual physical space.

[0101] Step S1415: Calculate the maximum length, maximum width, and total area of ​​all connected components, and take the maximum value as the maximum length, maximum width, and total area of ​​the overall damaged region.

[0102] After calculating the maximum length, maximum width, and total area of ​​each connected component, these parameters are statistically analyzed for all connected components. To obtain the maximum length, maximum width, and total area of ​​the overall damaged region, the maximum value of these parameters among all connected components is taken. For multiple independent damaged regions on the pier surface, their maximum length, maximum width, and total area are compared, and the maximum value is selected as the corresponding parameter for the overall damaged region. This ensures that the parameters of the overall damaged region reflect the extent and size of the damage under the most severe conditions. After organizing these maximum values, they serve as the basis for calculating the proportion of damaged areas. Combined with the standard image size of the corresponding structural parts, the proportion of damaged areas can be accurately calculated.

[0103] Step S147: Perform correlation analysis on the data of the proportion of the damaged area and the data of the trend of the damage propagation rate to determine whether the damage propagation rate accelerates as the damaged area increases.

[0104] After obtaining the data on the proportion of damaged areas and the trend of damage expansion rate, correlation analysis is needed to determine whether the damage expansion rate accelerates as the damaged area increases. First, a two-dimensional data set is established, containing the proportion of damaged areas and the corresponding expansion rate at different time points. The correlation coefficient is calculated by analyzing the distribution and variation patterns of these data points. The correlation coefficient represents the degree of linear correlation between the proportion of damaged areas and the expansion rate. If the correlation coefficient is greater than a preset threshold, the damage expansion rate is determined to accelerate as the damaged area increases; if the correlation coefficient is less than or equal to the preset threshold, the damage expansion rate is determined to have no significant correlation with the size of the damaged area. Combining the determination results with the location information of key damaged areas, damaged areas that simultaneously meet the criteria of large area proportion, accelerated expansion rate, and location near critical structural components are selected as areas requiring focused attention. For bridge pier damage, correlation analysis is performed on the proportion of damaged areas and the trend of damage expansion rate. Based on the analysis results, key damaged areas are selected, and the determination results and information on key damaged areas are integrated into the current damage status information, thereby enabling a more accurate assessment of the current damage status.

[0105] Step S150: Generate a road and bridge structure health diagnosis result based on the current damage status information and the damage development trend information, combined with the preset health level judgment rules.

[0106] Step S151: Extract the percentage of damaged area, damage type, location of key damaged areas and severity description from the current damage status information as static evaluation parameters.

[0107] After obtaining information on the current damage status and damage development trend, relevant parameters are first extracted from the current damage status information as static assessment parameters to generate structural health diagnosis results for the road and bridge. The proportion of damaged areas reflects the percentage of damage in the structural components; damage types can be categorized into different types such as cracks, peeling, and deformation; the location of key damage areas clarifies the specific locations of damage requiring focused attention; and the severity description provides a qualitative or quantitative assessment of the damage's severity. Taking bridge pier damage as an example, the proportion of damaged areas, the damage type being cracks, the location of key damage areas at a specific location on the pier, and the corresponding severity description are extracted and used as static assessment parameters. These static assessment parameters reflect the current damage status of the road and bridge structure.

[0108] Step S152: Extract the future damage location range, predicted expansion rate, and a list of key structural components that may be affected from the damage development trend information, as dynamic evaluation parameters.

[0109] Next, relevant parameters are extracted from the damage development trend information as dynamic evaluation parameters. The future damage location range indicates the area where the damage may expand in the future, the predicted expansion rate reflects the future speed of damage development, and the list of potentially affected key structural components lists the important parts that the damage may affect. For pier damage, the predicted location range that the future damage may cover, the predicted expansion rate, and the list of potentially affected key structural components such as the bearing connection area and the mid-span area of ​​the beam are extracted and used as dynamic evaluation parameters. These dynamically evaluated parameters can reflect the future development trend of the damage.

[0110] Step S153: According to the preset health level determination rules, assign weights to the static evaluation parameters and the dynamic evaluation parameters. The weights are determined based on the degree of influence of the damage type on structural safety.

[0111] After determining the static and dynamic evaluation parameters, weights are assigned to these parameters according to the preset health level judgment rules. Different damage types have varying degrees of impact on the safety of road and bridge structures; therefore, the weight allocation must be determined based on the damage type. For damage types that significantly impact structural safety, such as severe cracks and large-area spalling, the corresponding evaluation parameters should be assigned higher weights; while for damage types that have a smaller impact on structural safety, the corresponding evaluation parameter weights can be relatively lower. For bridge pier damage, if the damage type is severe cracks, then the static and dynamic evaluation parameters related to this damage type may be assigned higher weights. By rationally allocating weights, the impact of different damage types and evaluation parameters on the health status of road and bridge structures can be more accurately reflected.

[0112] Step S154: Calculate the comprehensive health score, which is obtained by weighted summation of the standardized quantified values ​​of the static assessment parameters and the dynamic assessment parameters.

[0113] After assigning weights to the static and dynamic evaluation parameters, a comprehensive health score is calculated. First, the static and dynamic evaluation parameters need to be standardized and quantified to make them comparable. Standardization and quantification transform parameters of different types and ranges into a unified numerical range, facilitating weighted summation. Then, based on the assigned weights, the standardized and quantified static and dynamic evaluation parameters are weighted and summed to obtain the comprehensive health score. For bridge pier damage, the standardized and quantified static evaluation parameters, such as the proportion of damaged area and damage type, and the dynamic evaluation parameters, such as the predicted future damage location and propagation rate, are weighted and summed according to their assigned weights to obtain the comprehensive health score for that bridge pier. This comprehensive health score can comprehensively reflect the current damage status and future development trend of the road and bridge structure.

[0114] Step S155: Determine the health level of the target road and bridge structure based on the matching result of the comprehensive health score and the preset health level threshold. The health level includes health level, attention level and warning level.

[0115] Step S1551: Obtain a preset health level threshold, wherein the health level threshold includes a scoring range corresponding to the health level, a scoring range corresponding to the attention level, and a scoring range corresponding to the warning level.

[0116] After obtaining the comprehensive health score, to determine the health level of the target road and bridge structure, it is necessary to acquire preset health level thresholds. These preset health level thresholds include scoring intervals corresponding to the health level, the attention level, and the warning level. These scoring intervals are pre-set based on the safety and reliability requirements of the road and bridge structure, with different scoring intervals corresponding to different health levels. For the health assessment of bridge piers, the preset health level thresholds are acquired to clarify the scoring range corresponding to each health level.

[0117] Step S1552: Compare the comprehensive health score with the scoring range corresponding to the health level. If the comprehensive health score falls within the scoring range corresponding to the health level, then the health level is determined to be healthy.

[0118] The calculated comprehensive health score is compared with the scoring range corresponding to the health level. If the comprehensive health score falls within the scoring range corresponding to the health level, it indicates that the damage status and development trend of the road and bridge structure are within an acceptable range, and the health level of the target road and bridge structure is determined to be healthy. For bridge piers, if their comprehensive health score falls within the scoring range corresponding to the health level, then the bridge pier can be determined to be in a healthy state, requiring only routine inspections and maintenance.

[0119] Step S1553: If the comprehensive health score is not within the scoring range corresponding to the health level, then compare the comprehensive health score with the scoring range corresponding to the attention level. If it is within the scoring range corresponding to the attention level, then determine the health level as attention.

[0120] If the overall health score does not fall within the scoring range corresponding to the health level, it is compared with the scoring range corresponding to the concern level. If the overall health score falls within the scoring range corresponding to the concern level, it indicates that the road and bridge structure has some damage or potential risk, but has not yet reached the point requiring emergency treatment. In this case, the health level of the target road and bridge structure is determined to be of concern. For bridge piers, if their overall health score is not within the scoring range of the health level but falls within the scoring range of the concern level, then the bridge pier needs to be monitored more closely and inspected regularly to detect the development and changes of damage in a timely manner.

[0121] Step S1554: If the comprehensive health score is not within the score range corresponding to the attention level, then the health level is determined to be a warning.

[0122] If the overall health score falls outside the range corresponding to either the health level or the concern level, the health level of the target road / bridge structure can be determined as a warning. This indicates that the damage to the road / bridge structure is relatively severe, and the development trend may significantly impact the structural safety, requiring immediate intervention. For bridge piers, if their overall health score exceeds the concern level's range, a warning signal should be issued, and professionals should be promptly arranged to conduct detailed inspections and assessments, and appropriate repair plans should be developed.

[0123] Step S1555: Based on the list of potentially affected key structural components in the damage development trend information, the health level is revised. If the damage may affect key structural components, the health level is adjusted upward by one level.

[0124] After determining the initial health level, the health level is revised based on the list of potentially affected critical structural components from the damage development trend information. If the damage may affect critical structural components, such as the bearing connection area or the mid-span area of ​​the beam, damage to these critical structural components would have a significant impact on the overall safety of the road and bridge structure; therefore, the health level needs to be adjusted upwards by one level. For bridge piers, if the damage development trend indicates that it may affect the bearing connection area, and the initially determined health level is at the concern level, then the health level is adjusted upwards to the warning level. Through this revision method, the actual health status of the road and bridge structure can be more accurately reflected, ensuring that sufficient attention is paid to damage that may affect critical structural components.

[0125] Step S1556: Use the corrected health level as the final health level determination result.

[0126] After revising the health level, the revised health level will be used as the final health level assessment result. This final health level assessment result can accurately reflect the health status of the target road and bridge structure. For bridge piers, based on the revised health level, such as the warning level, relevant departments can take timely and appropriate measures, such as emergency repairs and enhanced monitoring, to ensure the safety and normal use of the road and bridge structure.

[0127] Step S156: Extract the maintenance measures corresponding to the health level. The maintenance measures include the routine inspection plan corresponding to the health level, the local reinforcement suggestions corresponding to the attention level, and the emergency repair plan corresponding to the warning level.

[0128] After determining the final health level, the corresponding maintenance measures are extracted. Different health levels correspond to different maintenance measures to ensure the safety and reliability of the road and bridge structure. For a healthy level, the corresponding maintenance measures are usually routine inspection plans, regularly checking the road and bridge structure to promptly identify potential problems. For a concern level, local reinforcement recommendations may be needed to reinforce damaged areas or potentially affected parts to prevent further damage. For a warning level, an emergency repair plan needs to be implemented immediately to repair the damage and ensure the safety of the road and bridge structure. For bridge piers, if the final health level is determined to be concern, then corresponding local reinforcement recommendations are extracted, such as reinforcing the damaged parts of the pier to improve its load-bearing capacity and stability.

[0129] Step S157: Integrate the current damage status information, the damage development trend information, the health level, and the maintenance measures to generate the road and bridge structure health diagnosis result.

[0130] Finally, the current damage status information, damage development trend information, health level, and corresponding maintenance measures are integrated to generate a complete health diagnosis result for the road and bridge structure. This health diagnosis result includes important information such as the current damage status, future development trend, health level, and corresponding maintenance measures of the road and bridge structure. For bridge piers, the current damage status information, such as the current proportion of damaged area and damage type, the damage development trend information, such as the predicted location range and expansion rate of future damage, the finally determined health level, and the corresponding maintenance measures are integrated to form a complete health diagnosis result for the bridge piers. Relevant departments can formulate reasonable maintenance plans based on this diagnosis result to ensure the long-term safe operation of the road and bridge structure.

[0131] Figure 2 The illustration shows exemplary hardware and software components of an image analysis-based road and bridge structure health diagnostic system 100 that can implement the ideas of this application, according to some embodiments of this application. For example, a processor 120 can be used in the image analysis-based road and bridge structure health diagnostic system 100 and to perform the functions in this application.

[0132] The image analysis-based road and bridge structure health diagnosis system 100 can be a general-purpose server or a special-purpose server; both can be used to implement the image analysis-based road and bridge structure health diagnosis method of this application. Although only one server is shown in this application, for convenience, the functions described in this application can be implemented in a distributed manner on multiple similar platforms to balance the load.

[0133] For example, the image analysis-based road and bridge structure health diagnostic system 100 may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and various forms of storage media 140, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the image analysis-based road and bridge structure health diagnostic system 100 may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The methods of this application can be implemented according to these program instructions. The image analysis-based road and bridge structure health diagnostic system 100 also includes an I / O interface 150 between the computer and other input / output devices.

[0134] For ease of explanation, only one processor is described in the image analysis-based road and bridge structural health diagnosis system 100. However, it should be noted that the image analysis-based road and bridge structural health diagnosis system 100 of this application may also include multiple processors. Therefore, the steps performed by one processor described in this application may also be performed jointly or individually by multiple processors. For example, if the processor of the image analysis-based road and bridge structural health diagnosis system 100 performs steps A and B, it should be understood that steps A and B may also be performed jointly by two different processors or individually by one processor. For example, the first processor performs step A, the second processor performs step B, or the first processor and the second processor jointly perform steps A and B.

[0135] Furthermore, this embodiment of the invention also provides a readable storage medium, wherein computer-executable instructions are preset in the readable storage medium, and when the processor executes the computer-executable instructions, the above-mentioned method for road and bridge structure health diagnosis based on image analysis is implemented.

[0136] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A method for diagnosing the health of road and bridge structures based on image analysis, characterized in that, The method includes: Acquire a set of surface images of the target road and bridge structure, wherein the set of surface images includes surface images of the same structural part at different monitoring time points and different observation angles; The surface image set is spatiotemporally aligned to generate a spatiotemporally synchronized structural image set; The pre-trained damage feature association model is used to extract and analyze the structural image group to generate a damage feature association set. The damage feature association set includes cross-view spatial damage features, cross-time point temporal damage features, and the association features of the cross-view spatial damage features and cross-time point temporal damage features. Based on the damage feature association set, perform joint damage status analysis to generate current damage status information and damage development trend information; Based on the current damage status information and the damage development trend information, and combined with the preset health level determination rules, a road and bridge structure health diagnosis result is generated.

2. The method for road and bridge structural health diagnosis based on image analysis according to claim 1, characterized in that, The step of extracting and analyzing features from the structural image set using a pre-trained damage feature association model to generate a damage feature association set includes: Images from different perspectives at the same monitoring time point in the structural image group are input into the spatial feature extraction module of the damage feature association model. The spatial feature extraction module performs convolution operations on the images from each perspective through a convolutional network to extract the damage contour features from each perspective. The damage contour features include pixel-level features of crack boundaries, peeling area edges, and deformation area edges. Cross-view consistency verification is performed on the damage contour features under each viewpoint. By comparing the overlapping regions of features, damage contours that appear repeatedly in at least two views are selected to generate cross-view spatial damage features. The cross-view spatial damage features include information on the stable damage location, shape and coverage. Images from the same viewpoint at different time points in the structural image group are input into the temporal feature extraction module of the damage feature association model. The temporal feature extraction module calculates the pixel difference of the damage contour at adjacent time points through continuous frame difference operation, extracts the boundary movement trajectory of the damage expansion area, and generates cross-time point temporal damage features. The cross-time point temporal damage features include time series information of damage length growth path, width expansion direction, and area increase rate. The cross-view spatial damage features and the cross-time point temporal damage features are input into the association analysis module of the damage feature association model. The association analysis module establishes the correspondence between the spatial damage location and the temporal expansion trajectory through feature location matching, and generates a damage feature association set. The association set contains the association rules between the expansion path, expansion rate and initial damage scale of the damage location over time. The cross-view spatial damage features, the cross-time point temporal damage features, and the associated features are subjected to feature enhancement processing. The pixel response of the damage boundary is enhanced by local contrast adjustment, the background interference in the non-damaged area is suppressed, and the damage feature association set is finally output.

3. The method for health diagnosis of road and bridge structures based on image analysis according to claim 2, characterized in that, The process of performing cross-view consistency verification on the damage contour features under each viewpoint, and filtering out damage contours that appear repeatedly in at least two viewpoints by comparing overlapping feature regions, generates cross-view spatial damage features, including: The damage contour features from each viewpoint are converted into binary feature maps, where the pixel value of the damaged area is 1 and the pixel value of the non-damaged area is 0. The binarized feature maps are superimposed at the pixel level to obtain a superimposed feature map, where the value of each pixel in the superimposed feature map is the number of viewpoints covering that pixel; Pixel regions with values ​​greater than or equal to 2 in the superimposed feature map are extracted as damage regions consistent across viewpoints. A morphological closing operation is performed on the damage area with consistent cross-viewpoint to fill the small holes in the damage area and smooth the edges to obtain a continuous damage contour. Calculate the minimum bounding rectangle of the damage profile, and record the coordinates of the boundary vertices, width, and height of the minimum bounding rectangle as a quantitative description of the damage location and coverage area; The shape information of the damage contour and the quantitative description are organized into the cross-view spatial damage feature; Furthermore, the temporal feature extraction module calculates the pixel differences of the damage contour at adjacent time points through continuous frame differencing operations, and extracts the boundary movement trajectory of the damage expansion region, including: Pixel-level difference is performed on the damage contour feature maps of adjacent time points under the same viewpoint to obtain a difference feature map. The area with a pixel value of 1 in the difference feature map represents newly added damage or disappeared damage. The region with a pixel value of 1 in the differential feature map and located outside the original damage contour is retained as the newly added damage extension region; Extract the coordinates of the boundary points of the newly added damage expansion area, and record the displacement direction and distance of each boundary point relative to the damage contour boundary at the previous time point; The displacement data of the boundary points are statistically analyzed to calculate the average displacement direction and average displacement distance, which are used as the main direction and average rate of damage propagation. Track the coordinate changes of a set boundary point of the same damage profile at different time points and generate the movement trajectory curve of that set boundary point; The average displacement direction, average displacement distance, and movement trajectory curve are organized into the time-damage characteristics across time points.

4. The method for health diagnosis of road and bridge structures based on image analysis according to claim 1, characterized in that, The step of performing joint damage state analysis based on the damage feature association set to generate current damage state information and damage development trend information includes: Cross-view spatial damage features are extracted from the damage feature association set. Damaged and non-damaged regions are separated by an image segmentation algorithm. The maximum length, maximum width and total area of ​​the damaged region are calculated. Combined with the standard image size of the corresponding structural part, damage region proportion data is generated. Time-series damage features across time points are extracted from the damage feature association set. The historical average value of the damage propagation rate is calculated through time series analysis. Combined with the propagation rate at the current time point, data on the changing trend of the damage propagation rate is generated. Correlation analysis is performed on the data on the proportion of the damaged area and the trend data on the rate of damage expansion to determine whether the rate of damage expansion accelerates as the damaged area increases, and key damaged areas that expand rapidly are screened out. Based on the location, damage type, current area proportion, and expansion rate of the key damaged area, current damage status information is generated, which includes damage location coordinates, type identifier, and severity description. By using the damage propagation path prediction algorithm, based on the propagation trajectory information in the damage feature association set, the damage profile at future monitoring time points is simulated, the new areas that the damage may cover and the direction of change in the propagation rate are predicted, and damage development trend information is generated. The damage development trend information includes the future location range of the damage, the predicted value of the propagation rate, and the key structural components that may be affected.

5. The method for road and bridge structural health diagnosis based on image analysis according to claim 4, characterized in that, The step of separating the damaged region from the non-damaged region using an image segmentation algorithm and calculating the maximum length, maximum width, and total area of ​​the damaged region includes: The image region corresponding to the cross-view spatial damage feature is segmented by grayscale threshold, and the region with pixel grayscale value greater than the damage threshold is marked as the damaged region, and the rest is the non-damaged region. All connected components in the damaged region are identified using a connected component analysis algorithm, with each connected component representing an independent damaged region. For each connected component, calculate the minimum bounding rectangle of its contour, where the length of the long side of the minimum bounding rectangle is the maximum length of the damaged region and the length of the short side is the maximum width of the damaged region. The number of pixels in each connected component is counted, and combined with the resolution of the image acquisition device, the number of pixels is converted into the actual area, which is used as the total area of ​​the damaged region. The maximum length, maximum width, and total area of ​​all connected components are statistically analyzed, and the maximum value is taken as the maximum length, maximum width, and total area of ​​the overall damaged region. The maximum length, maximum width, and total area are used as the basis for calculating the proportion of the damaged area.

6. The method for health diagnosis of road and bridge structures based on image analysis according to claim 4, characterized in that, The damage propagation path prediction algorithm, based on the propagation trajectory information in the damage feature association set, simulates the damage contour at future monitoring time points and predicts the new areas that the damage may cover and the direction of change in the propagation rate, including: Extract historical data of damage propagation trajectory from the damage feature association set, wherein the historical data includes the coordinate sequence of damage boundary points at different time points; The coordinate sequence is fitted using a polynomial fitting algorithm to generate motion equations for the damaged boundary points, which describe the changes in the position of the boundary points over time. Based on the equation of motion, extrapolate the boundary point coordinates of future monitoring time points to generate the predicted boundary of the future damage profile; Calculate the area difference between the predicted boundary and the current damage contour, and combine it with the historical variation pattern of the damage propagation rate to predict the increment or decrease of the future damage propagation rate. Based on the location information of the predicted boundary, it is determined whether the damage extends to the critical structural components, and a list of potentially affected critical structural components is generated. The critical structural components include the support connection area and the mid-span area of ​​the beam. The predicted boundary coordinates, predicted propagation rate, and list of key structural components are compiled into the damage development trend information.

7. The method for health diagnosis of road and bridge structures based on image analysis according to claim 4, characterized in that, The step of performing correlation analysis on the data of the proportion of the damaged area and the trend data of the damage propagation rate to determine whether the damage propagation rate accelerates as the damaged area increases includes: A two-dimensional data point set is established to reflect the proportion of damaged areas and the rate of expansion. The two-dimensional data point set includes the proportion of damaged areas and the corresponding rate of expansion at different time points. Calculate the correlation coefficient of the two-dimensional data point set, whereby the correlation coefficient represents the degree of linear correlation between the proportion of damaged areas and the rate of expansion; If the correlation coefficient is greater than a preset threshold, it is determined that the damage propagation rate accelerates as the damage area increases; If the correlation coefficient is less than or equal to a preset threshold, it is determined that the damage propagation rate and the size of the damage area are not significantly correlated. Based on the judgment results and the location information of the key damage areas, damage areas that simultaneously meet the criteria of large area proportion, accelerated expansion rate and location near critical structural components are selected as damage areas that require special attention. The determination results and the information on the key damaged areas are integrated into the current damage status information.

8. The method for health diagnosis of road and bridge structures based on image analysis according to claim 1, characterized in that, The step of generating a road and bridge structure health diagnosis result based on the current damage status information and the damage development trend information, combined with preset health level determination rules, includes: The percentage of damaged areas, damage type, location of key damaged areas, and severity description are extracted from the current damage status information and used as static evaluation parameters. Extract the future damage location range, propagation rate prediction, and a list of key structural components that may be affected from the damage development trend information, as dynamic evaluation parameters; According to the preset health level determination rules, weights are assigned to the static evaluation parameters and the dynamic evaluation parameters, and the weights are determined based on the degree of influence of the damage type on structural safety. A comprehensive health score is calculated, which is obtained by weighted summation of the standardized quantified values ​​of the static assessment parameters and the dynamic assessment parameters; Based on the matching result between the comprehensive health score and the preset health level threshold, the health level of the target road and bridge structure is determined, and the health level includes health level, attention level and warning level; Extract the maintenance measures corresponding to the health level, which include the routine inspection plan corresponding to the health level, the local reinforcement suggestions corresponding to the attention level, and the emergency repair plan corresponding to the warning level. The current damage status information, the damage development trend information, the health level, and the maintenance measures are integrated to generate the road and bridge structure health diagnosis result.

9. The method for health diagnosis of road and bridge structures based on image analysis according to claim 8, characterized in that, The step of determining the health level of the target road and bridge structure based on the matching result of the comprehensive health score and the preset health level threshold includes: Obtain a preset health level threshold, which includes a scoring range corresponding to the health level, a scoring range corresponding to the attention level, and a scoring range corresponding to the warning level. The comprehensive health score is compared with the scoring range corresponding to the health level. If the comprehensive health score falls within the scoring range corresponding to the health level, the health level is determined to be healthy. If the overall health score is not within the scoring range corresponding to the health level, then the overall health score is compared with the scoring range corresponding to the concern level. If it is within the scoring range corresponding to the concern level, then the health level is determined to be concern. If the overall health score is not within the score range corresponding to the attention level, the health level is determined to be a warning. Based on the list of potentially affected key structural components in the damage development trend information, the health level is revised. If the damage may affect key structural components, the health level is adjusted upward by one level. The revised health level will be used as the final health level determination result.

10. A road and bridge structural health diagnosis system based on image analysis, characterized in that, The system includes a processor and a memory, the memory being connected to the processor. The memory is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the memory to implement the image analysis-based road and bridge structure health diagnosis method as described in any one of claims 1-9.

Images