AR-based bridge crack detection and analysis method and system

Abstract

The invention provides an AR-based bridge crack detection analysis method and system, and relates to the technical field of bridge detection analysis, and the technical scheme is characterized in that the method comprises the steps: monitoring the ambient light parameters of a bridge detection region; adjusting illumination output to stabilize imaging illumination conditions of the crack region; acquiring a crack image, and acquiring polarization information of the crack; stokes parameters are calculated, and a polarization degree image and a polarization angle image are generated; extracting a crack two-dimensional contour based on the polarization degree image and the polarization angle image; fusing the two-dimensional contour of the crack with the depth data to generate a three-dimensional model of the crack; and based on the crack three-dimensional model, crack geometric parameter quantitative analysis and multi-stage evolution state evaluation are carried out. The AR-based bridge crack detection and analysis method and system provided by the invention have the advantages of overcoming the influence of outdoor complex illumination conditions on image acquisition quality, improving the accuracy of crack recognition and three-dimensional modeling, and supporting reliable multi-stage evolution state evaluation.

Description

Technical Field

[0001] The present application relates to fields such as bridge detection and analysis, and specifically to an AR-based bridge crack detection and analysis method and system. Background Art

[0002] As a vital component of transportation infrastructure, the structural health of bridges is directly linked to public safety and economic performance. Over the long term, concrete bridges inevitably develop cracks due to a variety of factors, including material degradation, environmental erosion, and repeated traffic loads. The appearance and development of these cracks are crucial for assessing bridge structural performance, predicting their remaining service life, and developing effective maintenance strategies. Therefore, timely and accurate detection and quantitative analysis of bridge cracks are of paramount importance.

[0003] Traditional bridge crack detection methods rely primarily on manual visual inspection and measurement using tools such as crack width gauges. This approach has numerous limitations, including significant subjective influence on detection results, low efficiency, difficulty inspecting inaccessible or high-altitude areas, and a low level of informatization of detection data, hindering long-term tracking and management.

[0004] In recent years, with the rapid development of computer vision, sensor technology, and augmented reality (AR) technology, image processing-based nondestructive testing technology has been increasingly widely used in the field of bridge inspection. AR technology provides a new auxiliary means for bridge inspection by overlaying virtual information onto real-world scenes. For example, inspectors can wear AR head-mounted display devices integrated with depth cameras and high-definition cameras, use the AR system to load the bridge's building information model (BIM) as a reference, and use simultaneous localization and mapping (SLAM) technology to achieve precise alignment of the AR device's posture with the BIM model, thereby assisting inspectors in quickly locating and observing cracks on the bridge surface. When the inspector focuses on the target crack, the AR system can initiate a 3D scanning and reconstruction process to obtain the crack's 3D geometric parameters.

[0005] However, existing technologies still face significant challenges when applying AR technology to the specific scenario of refined three-dimensional modeling and quantitative analysis of outdoor bridge cracks. Outdoor bridge inspections are typically conducted in natural environments, where lighting conditions are complex and subject to dramatic dynamic changes. Factors such as real-time changes in the sun's angle, rapid cloud movement, and occlusion can cause significant and unpredictable changes in the illumination intensity and shadow distribution on the crack surface over a short period of time, severely impacting the quality and stability of image acquisition. Furthermore, the concrete bridge surface itself may exhibit uneven color, stains, water spots, and natural textures similar to crack features. The visual contrast between these surface features and subtle cracks is often low, especially in poor lighting conditions. Crack features can easily be obscured or confused with the background, increasing the difficulty of crack identification and extraction. Furthermore, uncontrollable temporary dynamic obstructions may exist during the inspection process, such as passing vehicles, construction equipment, or the movement of the inspector themselves. These obstructions can cause the loss or contamination of key information in the continuous image sequence.

[0006] The combined effect of the above-mentioned unfavorable factors makes it easy for AR system positioning algorithms (such as SLAM algorithms) that rely on continuous visual information to produce cumulative errors and drift, which in turn leads to problems such as morphological distortion, inaccurate measurement of dimensional parameters, model defects, or the introduction of false features in the subsequent three-dimensional reconstruction model of cracks based on image sequences (such as the SFM algorithm based on motion recovery). The core requirements of AR-assisted bridge inspection are not only to assist in positioning and observation, but also to be able to accurately measure the three-dimensional geometric parameters of the cracks, such as width, depth, and length, at the millimeter level, and to obtain high-precision three-dimensional crack models at different inspection cycles, so as to perform reliable quantitative comparative analysis under a unified benchmark, thereby accurately evaluating the true evolution state of the cracks. Existing technologies still have obvious deficiencies in dealing with the above-mentioned complex outdoor environmental factors, ensuring the accuracy of the reconstruction of the three-dimensional crack model, and the consistency of multi-period comparative analysis.

[0007] In view of the above problems, the existing technology is in urgent need of improvement. Summary of the Invention

[0008] The purpose of this application is to provide an AR-based bridge crack detection and analysis method and system, which has the advantages of overcoming the influence of complex outdoor lighting conditions on image acquisition quality, improving the accuracy of crack identification and three-dimensional modeling, and supporting reliable multi-period evolution state evaluation.

[0009] In the first aspect, this application provides an AR-based bridge crack detection and analysis method, the technical solution is as follows:

[0010] Monitor the ambient light parameters in the bridge inspection area;

[0011] adjusting the lighting output according to the ambient light parameters to stabilize the imaging lighting conditions of the crack area;

[0012] Crack images are collected at different polarization angles under stable imaging illumination conditions in the crack area to obtain polarization information of the crack.

[0013] Calculating the Stokes parameters of the polarization information to generate a degree of polarization image and a polarization angle image;

[0014] Extracting a two-dimensional contour of the crack based on the polarization degree image and the polarization angle image;

[0015] fusing the two-dimensional crack profile with the depth data to generate a three-dimensional crack model;

[0016] Based on the three-dimensional fracture model, quantitative analysis of fracture geometric parameters and multi-stage evolution state evaluation are performed.

[0017] Furthermore, in the present application, the step of capturing crack images at different polarization angles under imaging illumination conditions in a stable crack region to obtain polarization information of the crack includes:

[0018] Inertial measurement unit data is recorded synchronously while acquiring each crack image frame at different polarization angles under imaging illumination conditions in a stable crack region;

[0019] Select a frame of image in the sequence as the reference frame;

[0020] By extracting and matching visual feature points between each frame image, combined with the inertial measurement unit data, estimating the pose transformation of each frame relative to the reference frame;

[0021] According to the posture transformation, geometric correction is performed on each frame of the crack image so that it is aligned with the reference frame at the pixel level;

[0022] The polarization information is calculated using the corrected multi-angle crack images.

[0023] Furthermore, in the present application, the step of extracting the two-dimensional contour of the crack based on the polarization degree image and the polarization angle image includes:

[0024] performing image enhancement processing on the polarization degree image and the polarization angle image to improve the contrast between the crack and the background;

[0025] Perform preliminary segmentation on the image after improving the contrast between cracks and background, and mark potential independent crack areas;

[0026] Calculating the main direction distribution of pixels in each segmented area based on the direction information in the polarization angle image;

[0027] identifying and separating intersection points of cross cracks based on the continuity and consistency of the distribution of the principal directions;

[0028] Extracting the two-dimensional crack profile for each identified independent crack region based on the intersection points of the identified and separated cross cracks;

[0029] The extracted two-dimensional contours of the cracks are numbered and marked to establish the spatial topological relationship of the cracks.

[0030] Furthermore, in the present application, the step of fusing the two-dimensional crack profile with the depth data to generate a three-dimensional crack model includes:

[0031] Place multiple marking points in the area around the crack;

[0032] Each time a crack image is collected, the three-dimensional coordinate information of the marking point is collected synchronously;

[0033] Establishing a local coordinate system based on the three-dimensional coordinate information of the marking point;

[0034] Converting the two-dimensional profile and depth data of the crack into the local coordinate system;

[0035] Register the converted 2D fracture profile with the depth data;

[0036] The registered two-dimensional crack profile is fused with the depth data to generate a three-dimensional crack model;

[0037] The three-dimensional crack model is spatially associated with the bridge BIM model to record the position information of the crack in the bridge structure.

[0038] Furthermore, in the present application, the step of fusing the registered two-dimensional crack profile with the depth data to generate a three-dimensional crack model includes:

[0039] Acquire multispectral images covering visible and near-infrared bands;

[0040] Calculating a bridge surface moisture distribution map and a reflection intensity map based on the multispectral image;

[0041] identifying a humidity change area and a light reflection area based on the humidity distribution map and the reflection intensity map;

[0042] Identifying an affected area based on the locations of the humidity change area, the light reflection area, and the registered two-dimensional crack profile;

[0043] Reconstruct the depth value of the affected area according to the geometric constraints of the registered two-dimensional crack contour to obtain the corrected depth data;

[0044] The corrected depth data is fused with the registered two-dimensional crack contour to generate a three-dimensional crack model.

[0045] Furthermore, in the present application, the step of reconstructing the depth value of the affected area according to the geometric constraints of the registered two-dimensional crack profile to obtain corrected depth data includes:

[0046] Divide the impact area into multiple scale levels;

[0047] Apply a depth interpolation algorithm with different resolution parameters for each scale level;

[0048] Starting from the coarsest scale, the depth reconstruction result is refined step by step;

[0049] At each scale level, the depth reconstruction results are adjusted according to the geometric constraints of the registered 2D crack profiles;

[0050] The depth reconstruction results after adjustment at each scale level are fused to generate the corrected depth data.

[0051] Furthermore, in the present application, the step of identifying and separating intersection points of cross cracks based on the continuity and consistency of the main direction distribution includes:

[0052] Calculating the directional distribution gradient of each pixel point based on the continuity and consistency of the main direction distribution to generate a directional gradient map;

[0053] Setting multiple gradient thresholds on the directional gradient map to form a multi-level directional gradient region;

[0054] Adaptive adjustment of angular resolution is applied to each directional gradient region to improve the angular resolution capability in small-angle intersection areas;

[0055] In the directional gradient area after the angular resolution is adaptively adjusted, the directional mutation points are detected and marked as intersection candidates;

[0056] Clustering the intersection point candidates and merging candidate points whose distance is less than a preset threshold;

[0057] The intersection points of cross cracks are identified and separated based on the merged results after clustering.

[0058] Furthermore, in the present application, the step of clustering the intersection point candidates and merging candidate points whose distance is less than a preset threshold includes:

[0059] Calculate the local density of the candidate distribution area of ​​the intersection point;

[0060] Dividing the candidate distribution area of ​​the intersection point into a high-density area and a low-density area according to the local density;

[0061] Setting a first merging threshold for the high-density area and a second merging threshold for the low-density area, wherein the second merging threshold is greater than the first merging threshold;

[0062] Adaptively cluster the intersection point candidates according to the first merging threshold and the second merging threshold to merge candidate points whose distance is less than a preset threshold.

[0063] Furthermore, in the present application, the steps of performing quantitative analysis of fracture geometric parameters and multi-stage evolution state evaluation based on the three-dimensional fracture model include:

[0064] Collect three-dimensional fracture model data at multiple time points;

[0065] Calculate the rate of change of crack geometric parameters between adjacent time points;

[0066] determining a crack evolution rate based on the change rate;

[0067] When the crack evolution speed exceeds a first preset threshold, the next detection cycle is shortened; when the crack evolution speed exceeds a second preset threshold, a real-time monitoring mode is triggered to obtain multiple periods of crack data, where the second preset threshold is greater than the first preset threshold;

[0068] Based on the time series analysis method, multi-period crack data is processed to quantitatively analyze the crack geometric parameters, thereby calculating the crack growth rate and acceleration;

[0069] Performing a multi-period evolution state assessment based on the crack growth rate and acceleration, thereby predicting the future evolution trend of the crack;

[0070] A risk assessment report is generated based on the future evolution trend of the cracks.

[0071] Secondly, this application also proposes an AR-based bridge crack detection and analysis system, including:

[0072] Monitoring module, used to monitor the ambient light parameters in the bridge detection area;

[0073] an adjustment module, configured to adjust the lighting output according to the ambient light parameters to stabilize the imaging lighting conditions of the crack area;

[0074] An acquisition module is used to collect crack images at different polarization angles under imaging illumination conditions in a stable crack region to obtain polarization information of the crack;

[0075] A calculation module, configured to calculate the Stokes parameter of the polarization information and generate a polarization degree image and a polarization angle image;

[0076] An extraction module, configured to extract a two-dimensional profile of a crack based on the polarization degree image and the polarization angle image;

[0077] A generation module, configured to fuse the two-dimensional crack profile with the depth data to generate a three-dimensional crack model;

[0078] The evaluation module is used to perform quantitative analysis of fracture geometric parameters and multi-stage evolution state evaluation based on the three-dimensional fracture model.

[0079] From the above, it can be seen that the AR-based bridge crack detection and analysis method and system provided in this application overcomes the impact of complex outdoor lighting conditions on image acquisition quality by monitoring ambient light and adjusting lighting, using polarization information to enhance contrast, extracting two-dimensional contours and fusing them with depth to generate a three-dimensional model, and performing analysis based on the three-dimensional model. It improves the accuracy of crack identification and three-dimensional modeling, and supports reliable multi-period evolution state evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 A schematic flow chart of a bridge crack detection and analysis method based on AR provided in this application.

[0081] Figure 2 This is a structural diagram of an AR-based bridge crack detection and analysis system provided in this application.

[0082] In the figure: 1. Monitoring module; 2. Adjustment module; 3. Acquisition module; 4. Calculation module; 5. Extraction module; 6. Generation module; 7. Evaluation module. DETAILED DESCRIPTION

[0083] The technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. The components of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0084] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0085] Traditional outdoor bridge inspection environments are subject to strong dynamic non-uniform illumination, low contrast between cracks and the background, and interference from temporary dynamic obstructions. These factors degrade image acquisition quality, affecting the stability of the AR system's accurate pose estimation. This, in turn, leads to morphological distortion in the 3D reconstructed crack model and reduced accuracy in dimensional parameter measurement. Consequently, achieving high-precision 3D quantitative analysis of bridge cracks and reliable assessment of their multi-stage evolution state are difficult.

[0086] For example, a user uses an AR device that integrates a visual sensor and an inertial measurement unit to inspect concrete bridge components. The system tracks the device's position using visual inertial odometry and uses the captured images to construct a local 3D environmental model. During the inspection process, changes in the sun's angle or cloud movement cause rapid changes in the intensity and direction of illumination on the component's surface. The inherent texture and stains on the concrete surface are not significantly different from the grayscale or color of fine cracks. At the same time, passing vehicles or pedestrians can briefly block the inspection area. These events lead to unstable feature point extraction and matching in continuous image sequences, and the visual odometry accumulates drift. As a result, the system's estimated device position deviates from its actual position, affecting the accuracy of subsequent 3D crack reconstruction based on multi-view images. The reconstructed 3D crack model may be locally distorted, have incorrect scale, or contain non-crack features, failing to accurately reflect the crack's true geometry.

[0087] In this regard, refer to Figure 1 , this application proposes a bridge crack detection and analysis method based on AR, including:

[0088] S110, monitoring ambient light parameters in the bridge detection area;

[0089] S120, adjusting the lighting output according to the ambient light parameters to stabilize the imaging lighting conditions of the crack area;

[0090] S130, capturing crack images at different polarization angles under imaging illumination conditions in a stable crack region to obtain polarization information of the crack;

[0091] S140, calculating the Stokes parameter of the polarization information to generate a polarization degree image and a polarization angle image;

[0092] S150, extracting a two-dimensional contour of the crack based on the polarization degree image and the polarization angle image;

[0093] S160, fusing the two-dimensional crack profile with the depth data to generate a three-dimensional crack model;

[0094] S170. Based on the three-dimensional fracture model, perform quantitative analysis of fracture geometric parameters and multi-stage evolution state evaluation.

[0095] Monitoring the ambient light parameters of the bridge inspection area involves acquiring real-time information about the intensity and direction of natural light in the inspection area. This can be achieved using light sensors. This information is used to understand the current lighting environment and provide a basis for subsequent lighting adjustments.

[0096] Adjusting lighting output based on ambient light parameters involves controlling the brightness and direction of the auxiliary light source based on the monitored ambient light information. This can be achieved using an adjustable light source and control circuit. This adjustment aims to offset the effects of ambient light variations, maintain stable lighting conditions in the crack area, and improve image acquisition quality.

[0097] Capturing crack images at different polarization angles under stable imaging lighting conditions within the crack region and obtaining crack polarization information involves capturing images of the crack region from multiple polarization directions using a polarization camera or rotating polarizer under stable lighting conditions. This can be achieved using a polarization camera or a camera with a rotating polarizer. This process captures polarization characteristics resulting from the interaction between light and the crack surface, which are robust to variations in lighting and surface texture.

[0098] Calculating the Stokes parameters of polarization information and generating degree of polarization and angle of polarization images involves calculating the Stokes parameters describing the polarization state of light based on images captured at different polarization angles, extracting the degree of polarization (DoP) and angle of polarization (AoP) information from these, and generating corresponding images. This can be achieved using image processing algorithms. These images intuitively demonstrate the distribution of polarization characteristics in the crack area, providing a data foundation for subsequent crack extraction based on polarization characteristics.

[0099] Extracting 2D crack contours from polarization and polarization angle images involves using the differences in polarization characteristics between the crack and the background in these images to identify and delineate the crack's 2D shape and location. This can be achieved using an image segmentation algorithm. This method leverages polarization information to enhance crack contrast, improve crack extraction accuracy, and reduce false detections.

[0100] Fusion of the 2D fracture profile and depth data to generate a 3D fracture model involves combining the extracted 2D fracture profile information with the 3D depth information of the same area to reconstruct the fracture geometry in 3D space. This can be achieved using a 3D reconstruction algorithm. This fusion process elevates the 2D planar information into 3D space, generating a fracture model with realistic size and spatial location.

[0101] Quantitative analysis of fracture geometry and multi-period evolutionary status assessment based on a 3D fracture model involves measuring geometric parameters such as fracture width, depth, and length using the generated 3D fracture model, and performing comparative analysis between models acquired at different time points. This can be achieved using 3D measurement and time series analysis algorithms. This analysis process enables precise quantification of the fracture state and reliable assessment of its evolution over time.

[0102] The core innovation of this application lies in combining active lighting control, polarization imaging technology and AR three-dimensional reconstruction process to overcome the impact of complex outdoor lighting and low-contrast environment on bridge crack detection, and realize high-precision three-dimensional modeling of cracks, quantitative analysis of geometric parameters and reliable evaluation of multi-period evolution status.

[0103] The working process and principle of this application is to monitor the ambient light parameters of the bridge inspection area to obtain the current lighting conditions in real time. Based on the monitored ambient light parameters, the lighting output is adjusted, for example, by controlling the brightness or direction of the auxiliary light source to stabilize the imaging lighting conditions in the crack area. This overcomes the impact of dynamically changing outdoor ambient light on image acquisition quality and ensures that subsequently processed images have a consistent lighting foundation.

[0104] Crack images are captured at different polarization angles under stable imaging lighting conditions within the crack area to obtain polarization information. Even under stable lighting, the contrast between the crack and the background can be low. Polarization imaging can capture the difference in light polarization characteristics between the crack and the surrounding area, enhancing the crack's characteristics.

[0105] The Stokes parameters of the acquired polarization information are calculated to generate polarization degree and polarization angle images. The Stokes parameters quantify the polarization state of light and convert the polarization information into images, providing a data foundation for subsequent crack extraction based on polarization characteristics. The two-dimensional crack contours are extracted based on the generated polarization degree and polarization angle images. Using polarization information for crack extraction is more robust to changes in illumination and surface texture, enabling more accurate identification and delineation of the two-dimensional shape and location of the crack. The extracted two-dimensional crack contours are fused with depth data to generate a three-dimensional crack model. Combining the two-dimensional crack contour information with the corresponding three-dimensional depth information allows the true shape and size of the crack to be reconstructed in three-dimensional space, generating a three-dimensional crack model with spatial coordinates. Based on the generated three-dimensional model, quantitative analysis of the crack's geometric parameters and multi-period evolution state assessment are performed. Accurate geometric dimension measurements and analysis are performed in three-dimensional space. Three-dimensional models acquired at different time points are compared to quantitatively analyze crack changes over time and assess the crack's evolutionary state.

[0106] As a preferred embodiment, the solution of the present application is specifically implemented as follows: an AR detection device that integrates an ambient light sensor, an adjustable light source, a polarization camera (or a camera with a rotatable polarizer) and a depth sensor is used. The ambient light sensor continuously monitors the ambient light intensity and direction of the bridge detection area. According to the reading of the ambient light sensor, the output power and illumination angle of the adjustable light source are controlled so that the illumination intensity and uniformity on the surface of the crack area are maintained within a preset range. After the lighting conditions are stabilized, the polarization camera collects images of the crack area at multiple preset polarization angles (for example, 0 degrees, 45 degrees, 90 degrees, 135 degrees). The Stokes parameters (S0, S1, S2, S3) of each pixel are calculated from these images with different polarization angles. Polarization degree images and polarization angle images are obtained based on the Stokes parameter calculations. Image processing algorithms, such as threshold segmentation, edge detection or texture analysis-based methods, are applied to the polarization degree images and polarization angle images to extract the two-dimensional pixel outline of the crack. The depth sensor is used simultaneously to obtain depth data of the same area. The extracted two-dimensional crack outline information is spatially matched and fused with the corresponding depth data to generate a point cloud or mesh model representing the three-dimensional morphology of the crack. Based on the generated 3D crack model, a 3D measurement algorithm is used to calculate the crack's geometric parameters, such as width, depth, length, and area. The 3D crack model generated during the current inspection cycle is spatially aligned and compared with the 3D model acquired and stored during the previous inspection cycle. This allows for quantitative analysis of how the crack's geometric parameters change over time and to assess crack expansion and deepening.

[0107] Through the above-mentioned solution, this application solves the issues of image acquisition quality degradation caused by strong dynamic non-uniform illumination, low contrast between cracks and background, and interference from temporary dynamic obstructions in outdoor bridge inspection environments. This ensures the stability of the AR system's accurate pose estimation, thereby improving the morphological accuracy of the 3D reconstructed crack model and the accuracy of dimensional parameter measurement. Ultimately, high-precision 3D quantitative analysis of bridge cracks and reliable assessment of their multi-stage evolution are achieved.

[0108] In some of the above-mentioned schemes of the present application, it is proposed to collect crack images at different polarization angles under imaging lighting conditions in a stable crack area to obtain polarization information of the cracks for subsequent crack analysis. However, in a complex environment such as an outdoor bridge, when collecting images at different polarization angles, since the camera may have slight movements and the scene may have slight changes, directly using these images to calculate the polarization information will result in inaccurate correspondence between pixel points, thereby affecting the accuracy of the polarization information, and reducing the accuracy of subsequent crack extraction and three-dimensional reconstruction based on polarization information.

[0109] In this regard, the present application further proposes to synchronously record inertial measurement unit data when collecting each frame of crack image at different polarization angles under imaging lighting conditions in a stable crack area; select a frame of image in the sequence as a reference frame; estimate the posture transformation of each frame relative to the reference frame by extracting and matching visual feature points between each frame of image, combined with inertial measurement unit data; according to the posture transformation, perform geometric correction on each frame of crack image so that it is aligned with the reference frame at the pixel level; and use the corrected multi-angle crack images to calculate polarization information.

[0110] Under stable imaging lighting conditions in the crack region, while capturing crack image sequences at different polarization angles, data from the inertial measurement unit (IMU) associated with the image acquisition device is simultaneously recorded. The IMU can include a three-axis accelerometer and a three-axis gyroscope, which measure the device's linear acceleration and angular velocity. This data provides information about the device's motion during image sequence acquisition. Furthermore, a single frame from the captured image sequence is selected as a reference, referred to as the base frame. All subsequent images are aligned to this base frame.

[0111] Each frame in an image sequence is processed against a reference frame to extract visual feature points, such as corners, edges, or scale-invariant features. These feature points exhibit a certain degree of robustness across different images. The extracted feature points are then matched to find feature pairs corresponding to the same spatial point in different images. Simultaneously with the visual feature point extraction and matching process, or as part of subsequent processing, synchronously recorded inertial measurement unit (IMU) data can be incorporated into the process to provide an initial estimate of camera motion and assist in the visual feature matching process. For example, IMU data can be used to narrow the search range by predicting the approximate location of feature points in the next frame or to filter out false visual matches. By fusing the visual feature matching results with the IMU data, the pose transformation of each frame in the sequence relative to the reference frame can be estimated—that is, the change in the camera's position and attitude relative to the reference frame at the time each frame was captured. For example, optimization-based methods can be used to combine the visual reprojection error and the IMU measurement error for joint optimization, resulting in more accurate pose estimates.

[0112] Based on the estimated pose transformation of each frame of image relative to the reference frame, these images are geometrically corrected. The process of geometric correction is to perform a two-dimensional transformation on each frame of image so that its content is aligned with the reference frame at the pixel level. This can be achieved through image resampling technology, mapping the pixels in the non-reference frame image to the image coordinate system of the reference frame based on the pose transformation relationship. For example, if a frame of image is estimated to have been translated and rotated relative to the reference frame, the image can be corrected through affine transformation or more complex projection transformation so that the same spatial point in the image has the same pixel coordinates in the corrected image and the reference frame. After geometric correction, the corresponding pixel points in the image sequence collected at different polarization angles represent the information of the same spatial point in the crack area.

[0113] Polarization information is calculated using these geometrically corrected, pixel-level aligned multi-angle crack images. Polarization information is usually represented by calculating the Stokes parameters, which requires at least three images with different polarization angles. For example, images with polarization angles of 0, 45, 90, and 135 degrees can be collected. The corrected images ensure that when calculating the Stokes parameters of each pixel point, the pixel values ​​of the different polarization angles used do correspond to the polarization state of the same point on the crack surface. In this way, the polarization degree image and polarization angle image of each pixel point can be accurately calculated. This accurate polarization information is then used in subsequent crack analysis steps, such as extracting the two-dimensional contour of the crack and generating a three-dimensional crack model. In this way, the accuracy of the polarization information calculation is improved, thereby improving the accuracy of subsequent crack analysis.

[0114] Specifically, under the lighting conditions of a stable crack area, a camera with a polarization filter and an inertial measurement unit is used to collect crack images. For example, the camera can switch the angle of the polarization filter in sequence, or use a camera with multiple polarization direction sensors. While collecting each frame of image (corresponding to a polarization angle), the inertial measurement unit records the current motion data. Suppose that a sequence of images with four polarization angles of 0°, 45°, 90° and 135° are collected. Select one frame from these four frames, such as the image with a polarization angle of 0°, as the reference frame. The remaining images with polarization angles of 45°, 90° and 135° are processed separately with the 0° reference frame.

[0115] Between each pair of images (e.g., a 45° image and a 0° reference frame), visual feature points are extracted and matched. For example, SIFT or ORB algorithms can be used to extract feature points, and FLANN or BFMatcher can be used for matching. At the same time, the collected inertial measurement unit data, such as the acceleration and angular velocity data at the time of collecting the 45° image, is used to assist in estimating the pose transformation of the 45° image relative to the 0° reference frame. This can be achieved through a visual inertial odometry (VIO) algorithm, which fuses visual and inertial data for pose estimation. As a result, the rotation and translation parameters of the 45° image relative to the 0° reference frame are obtained. The same process is repeated for the 90° and 135° images to estimate their pose transformation relative to the 0° reference frame.

[0116] Based on the estimated pose transformation parameters, the 45°, 90°, and 135° images are geometrically corrected. For example, if the 45° image is estimated to have a slight translation and rotation relative to the 0° reference frame, the corresponding two-dimensional transformation (such as an affine transformation) is applied to resample the 45° image so that it is aligned with the 0° reference frame on the pixel grid. Similar corrections are performed on the 90° and 135° images. After correction, the same pixel coordinates in the four images with different polarization angles (0°, corrected 45°, corrected 90°, and corrected 135°) represent the same point on the crack surface.

[0117] Using these corrected images, the Stokes parameters of each pixel are calculated based on the principles of polarization optics. For example, the Stokes parameters S0, S1, and S2 can be calculated using the following formulas: S0 = I(0) + I(90), S1 = I(0) - I(90), S2 = I(45) - I(135), where I(θ) represents the intensity value of the corresponding pixel in the corrected θ polarization angle image. Based on the Stokes parameters, the degree of polarization (DoP) and angle of polarization (AoP) of each pixel are further calculated to generate a degree of polarization image and an angle of polarization image. For example, DoP = sqrt(S1^2 + S2^2) / S0, AoP = 0.5*arctan2(S2, S1).

[0118] Through this process, even if the camera moves slightly or the scene changes subtly while capturing images at different polarization angles, the calculated Stokes parameters, polarization degree images, and polarization angle images can accurately reflect the true polarization information of the crack area due to the precise geometric correction and pixel alignment of the images. This overcomes the problem of precise pixel alignment when directly calculating polarization information using uncorrected images, improving the accuracy of polarization information. This provides a more reliable data foundation for subsequent crack extraction and 3D reconstruction based on polarization information, helping to improve the accuracy of 2D crack contour extraction and 3D crack models.

[0119] In some of the above-mentioned solutions of this application, it is proposed to extract the two-dimensional contour of the crack based on the polarization degree image and the polarization angle image to obtain the planar morphological information of the crack. However, in the complex environment of outdoor bridges, the bridge surface may have uneven color, stains, water stains, and natural textures similar to crack characteristics. The visual contrast between these features and fine cracks is low. Especially in poor lighting, the crack characteristics are easily submerged or confused, resulting in inaccurate or incomplete crack contours extracted based on polarization information. In particular, for cross cracks, the polarization information at their intersection may be complex and difficult to distinguish. Existing methods have difficulty in accurately identifying and separating the various branches of the cross crack, which affects the subsequent precise quantification of crack geometric parameters and the establishment of spatial topological relationships.

[0120] In this regard, the present application further proposes that the steps of extracting the two-dimensional contour of the crack based on the polarization degree image and the polarization angle image include:

[0121] Perform image enhancement processing on the polarization degree image and polarization angle image to improve the contrast between the crack and the background;

[0122] Perform preliminary segmentation on the image after improving the contrast between cracks and background, and mark potential independent crack areas;

[0123] Based on the direction information in the polarization angle image, the main direction distribution of the pixel points in each segmented area is calculated;

[0124] Identify and separate the intersection points of cross cracks based on the continuity and consistency of the main direction distribution;

[0125] Extracting the two-dimensional crack profile for each identified independent crack region based on the intersection points of the identified and separated cross cracks;

[0126] The extracted two-dimensional contours of the cracks are numbered and marked to establish the spatial topological relationship of the cracks.

[0127] Among them, this solution aims to solve the problem of how to extract the two-dimensional contour of cracks based on polarization information in complex environments, especially when there are low contrast and cross cracks.

[0128] First, image enhancement is performed on the polarization degree and polarization angle images to improve the contrast between the cracks and the background. Due to the complexity of the outdoor environment and the concrete surface itself, crack features in the original polarization images obtained by collecting images at different polarization angles and calculating the Stokes parameters may not be prominent enough. Enhancement makes the crack area more easily identified by subsequent algorithms, overcoming the challenge of low contrast. Image enhancement can be achieved using a variety of techniques, such as contrast stretching, histogram equalization, or specific filtering methods to highlight crack features.

[0129] Next, the image, after enhancing the contrast between cracks and background, undergoes preliminary segmentation, marking potential independent crack regions. This step groups pixels in the image that may represent cracks, forming a preliminary regional division and providing a foundation for subsequent detailed processing. This preliminary segmentation can be achieved by setting a threshold, applying a region growing algorithm, or using a texture analysis-based method. This divides the image into several regions that may contain cracks.

[0130] Then, based on the directional information in the polarization angle image, the principal direction distribution of the pixels within each segmented area is calculated. Polarization angle information is related to the normal direction of the object's surface. For linear cracks, the polarization angle will exhibit a certain directionality. Calculating the principal direction distribution can reveal the internal orientation of the crack region and provide a basis for distinguishing different crack branches. The principal direction distribution can be calculated based on the local gradient direction or by analyzing the changes in the polarization angle of the pixel in different directions. For example, the polarization angle gradient within the neighborhood of each pixel can be calculated and its principal direction can be counted.

[0131] Furthermore, based on the continuity and consistency of the main direction distribution, the intersection points of cross cracks are identified and separated. By analyzing the changes in the direction distribution of pixel points, areas with sudden changes in direction can be detected, which often correspond to the intersection points of cracks. For example, on a crack branch, the main direction distribution of pixel points is usually continuous and consistent, while at the intersection point, the direction distribution changes significantly. Identifying and separating these intersection points can decompose complex cross cracks into relatively independent crack segments. Methods for identifying intersection points can include analyzing directional gradient maps, detecting areas with high directional change rates, or applying graph theory methods to analyze connectivity within the region.

[0132] Subsequently, a 2D fracture contour is extracted for each identified independent fracture region based on the intersection points of the identified and separated fractures. After the intersection points are separated, each fracture segment can be treated as an independent region for contour extraction, avoiding confusion at the intersection points and improving the accuracy of contour extraction. Contour extraction can be achieved using edge detection algorithms, active contour models, or morphological methods.

[0133] Finally, the extracted two-dimensional fracture profiles are numbered and labeled to establish the spatial topological relationship of the fractures. Each extracted fracture profile is uniquely identified, and the connection relationship between them is recorded (for example, which fracture segments are connected at which intersection). This lays the foundation for subsequent quantitative analysis of fracture geometric parameters, multi-period evolution comparison, and spatial correlation and information management in the AR system. By utilizing the directional information of the polarization angle to process intersecting fractures, this solution can more effectively address the challenges of fracture extraction in complex environments, especially when the fracture contrast is low and complex intersecting structures are present.

[0134] In some of the aforementioned solutions in this application, it was proposed to fuse the 2D crack profile with depth data to generate a 3D crack model for subsequent quantitative analysis of geometric parameters and multi-period evolution state assessment. However, in complex outdoor environments, AR system positioning may drift, resulting in inaccurate depth data or inconsistency with the spatial relationship of the 2D profile. Direct fusion may generate a 3D model with distorted morphology and inaccurate parameters, which cannot meet the requirements of high-precision detection and reliable multi-period comparison.

[0135] In this regard, the present application further proposes to fuse the two-dimensional contour of the crack with the depth data, and the steps of generating a three-dimensional model of the crack include: arranging multiple marking points in the area around the crack; synchronously collecting the three-dimensional coordinate information of the marking points each time the crack image is collected; establishing a local coordinate system based on the three-dimensional coordinate information of the marking points; converting the two-dimensional contour of the crack and the depth data into the local coordinate system; aligning the converted two-dimensional contour of the crack with the depth data; fusing the aligned two-dimensional contour of the crack with the depth data to generate a three-dimensional model of the crack; spatially associating the three-dimensional crack model with the bridge BIM model to record the position information of the crack in the bridge structure.

[0136] Among them, multiple marking points are arranged in the area around the crack. These marking points can be reflective markers, QR code markers or markers with specific geometric shapes that are physically fixed on the surface of the bridge structure. These marking points maintain their relative positions unchanged during the detection cycle, providing a stable spatial reference. Each time the crack image is collected, the three-dimensional coordinate information of the marking points is collected synchronously. This can be achieved in a variety of ways, for example, using a depth sensor integrated with the image acquisition device to simultaneously obtain the depth information of the marking points, or reconstructing the three-dimensional position of the marking points through multi-view image acquisition combined with structured light or stereo vision technology, or using an independent measurement device (such as a total station) to accurately measure and record the three-dimensional coordinates of the marking points in advance.

[0137] A local coordinate system is established based on the 3D coordinate information of the marker points. Typically, at least three non-collinear marker points are selected as a reference to define the origin and coordinate axis directions of the local coordinate system. For example, one marker point can be set as the origin, while the other two marker points determine the direction of a plane or an axis. The 2D crack profile and depth data are converted to the local coordinate system. The 2D crack profile is typically acquired in the image coordinate system or the device coordinate system, while the depth data is typically acquired in the device coordinate system.

[0138] By using the known positional relationship of the device relative to the markers (this positional relationship can be calculated by identifying the markers in the image and using their known 3D coordinates), the 2D profile and depth data can be transformed from their original coordinate system to a local coordinate system established based on the markers. The transformed 2D crack profile and depth data are then registered. Within the same stable local coordinate system, various registration algorithms, such as feature-based registration or iterative closest point (ICP) algorithms, can be used to precisely align the crack shape described by the 2D profile with the surface geometry reflected by the depth data.

[0139] The registered 2D fracture profile is fused with the depth data to generate a 3D fracture model. This fusion process overlays or maps the precise boundaries and details (such as width variations) of the 2D profile onto the registered depth data, generating a 3D model that captures the precise fracture geometry.

[0140] For example, 2D contours can be used as constraints to locally correct or interpolate depth data to more accurately reflect the depth and profile of the crack. The 3D crack model is spatially associated with the bridge BIM model to record the crack's location within the bridge structure. This can be achieved by aligning the established local coordinate system with the global coordinate system of the bridge BIM model. For example, if the global coordinates of the marker points are known, the transformation matrix from the local coordinate system to the global coordinate system can be directly calculated. The resulting 3D crack model is then placed within the 3D context of the overall bridge structure, facilitating subsequent management, analysis, and visualization.

[0141] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0142] In a specific inspection area of ​​a concrete bridge, such as a bridge pier surface, multiple marking points are placed around the cracks to be detected. These marking points can be physical markers, such as reflective targets or identification points with unique patterns, and are firmly fixed to the bridge surface.

[0143] Each time a crack image is captured, an AR device equipped with a camera and a depth sensor is used. The device simultaneously captures images of the crack area and the three-dimensional coordinate information of the deployed markers. The three-dimensional coordinate information of the markers can be directly obtained through the depth sensor or obtained through other positioning or measurement modules integrated into the device. Based on the three-dimensional coordinate information of the captured markers, a local coordinate system is established. For example, one of the markers can be selected as the origin of the local coordinate system, and the direction and plane of the coordinate axis can be determined by the other two markers. The two-dimensional crack contour extracted from the crack image (usually in the image coordinate system or the device coordinate system) and the depth data of the captured crack area (usually in the device coordinate system) are converted into this local coordinate system established based on the markers.

[0144] In the local coordinate system, the converted 2D crack profile and depth data are registered. This can be achieved by projecting the 2D profile into 3D space and aligning it with the point cloud composed of the depth data. For example, using the Iterative Closest Point (ICP) algorithm or feature-based registration method, the registered 2D crack profile is fused with the depth data to generate a 3D model of the crack. During the fusion process, the precise boundary and shape information provided by the 2D profile can be used to correct or supplement the information in the details or missing areas of the depth data, thereby generating a more morphologically accurate 3D model.

[0145] Finally, the generated 3D crack model was spatially linked to the bridge's Building Information Model (BIM). This was achieved by determining the transformation relationship between the local coordinate system and the bridge's BIM global coordinate system, thereby accurately positioning and recording the crack model within the overall digital model of the bridge structure.

[0146] This technical solution addresses the problem of inaccurate depth data or inconsistency with the spatial relationship of the 2D outline caused by the AR system's positioning instability in complex outdoor environments. By placing markers in the area surrounding the crack and establishing a stable local coordinate system based on them, data acquisition, conversion, registration, and fusion are performed within a stable spatial reference that is unaffected by the AR system's instantaneous positioning drift. As a result, the 2D crack outline and depth data can be precisely aligned and fused, generating a morphologically accurate and parameter-reliable 3D crack model. This high-precision 3D model is then spatially associated with the bridge's BIM model, enabling precise recording of the crack's location within the bridge structure.

[0147] In some of the above-mentioned schemes in this application, it is proposed to fuse the two-dimensional contour of the crack with the depth data to generate a three-dimensional crack model. However, in a complex detection environment such as an outdoor bridge with strong dynamic non-uniform lighting, low contrast between the crack and the background, and interference from temporary dynamic obstructions, the collected depth data may be affected by factors such as humidity changes and light reflection, resulting in inaccurate depth values, especially in the crack and its surrounding areas. Directly fusing the affected depth data with the two-dimensional contour of the crack will cause the generated three-dimensional crack model to be morphologically distorted and the dimensional parameter measurement to be inaccurate, which makes it difficult to meet the needs of high-precision three-dimensional quantitative analysis and reliable evaluation of multi-period evolutionary states.

[0148] In this regard, the present application further proposes that the steps of fusing the registered two-dimensional crack profile with the depth data to generate a three-dimensional crack model include:

[0149] Acquire multispectral images covering visible and near-infrared bands;

[0150] Calculate the bridge surface moisture distribution map and reflection intensity map based on multispectral images;

[0151] Identify humidity change areas and light reflection areas based on humidity distribution maps and reflection intensity maps;

[0152] Identify the affected area based on the location of humidity change areas, light reflection areas, and the registered 2D crack profiles;

[0153] Reconstruct the depth value of the affected area according to the geometric constraints of the registered two-dimensional crack contour to obtain the corrected depth data;

[0154] The corrected depth data is fused with the registered two-dimensional crack contour to generate a three-dimensional crack model.

[0155] AR inspection equipment equipped with a multispectral camera can capture multispectral images encompassing both visible and near-infrared wavelengths. For example, a camera can simultaneously capture images in the visible light band (450-650 nanometers) and the near-infrared band (750-950 nanometers). The near-infrared band is sensitive to changes in moisture and surface properties of materials, and when combined with visible light, it can provide richer information than a single visible light image.

[0156] Calculating a bridge surface moisture distribution map and reflectance intensity map from multispectral imagery can be done by exploiting differences or ratios between image bands. For example, a moisture distribution map can be generated by calculating the absorbance ratio between a specific near-infrared band and the visible light band, while a reflectance intensity map can be generated based on pixel intensity values ​​in the visible light band or near-infrared band. Identifying areas of moisture variation and light reflection based on the moisture distribution and reflectance intensity maps can be accomplished by setting thresholds or applying image segmentation algorithms.

[0157] For example, areas above a certain threshold in the humidity distribution map are marked as humidity change areas, and areas above or below a certain threshold in the reflection intensity map are marked as light reflection areas. The affected areas are identified based on the locations of the humidity change areas, light reflection areas, and the aligned two-dimensional crack contours. This involves superimposing or performing logical judgment on the aforementioned identified environmental impact areas with the locations of the two-dimensional crack contours obtained through other steps (e.g., by placing markers around the crack and establishing a local coordinate system for alignment). For example, humidity change areas and light reflection areas that overlap or are adjacent to the two-dimensional crack contours are identified as affected areas.

[0158] After the two-dimensional crack contour position has been accurately positioned through registration, the two-dimensional contour is used as a reliable geometric boundary and shape reference to correct or interpolate and reconstruct the potentially inaccurate original depth data in the affected area. For example, the depth variation trend of the crack contour, crack width information, and reliable depth values ​​of the surrounding unaffected areas can be used to generate depth data that is more in line with the actual situation through interpolation, fitting, or model-based reconstruction algorithms. The corrected depth data is fused with the registered two-dimensional crack contour to generate a three-dimensional crack model. This is to combine the corrected depth information with the accurate two-dimensional contour to construct a three-dimensional model that reflects the true shape of the crack. This process can be achieved using technologies such as point cloud fusion and mesh reconstruction.

[0159] Through these steps, this solution adds a correction link for inaccurate depth data caused by environmental factors to the existing framework of fusing two-dimensional contours and depth data to generate three-dimensional models. In particular, it uses the aligned two-dimensional contours as geometric constraints for depth reconstruction, thereby improving the accuracy of generating three-dimensional crack models in complex outdoor environments.

[0160] In some of the above-mentioned schemes of this application, it is proposed to reconstruct the depth value of the affected area according to the geometric constraints of the aligned two-dimensional contour of the crack to obtain corrected depth data, thereby generating a more accurate three-dimensional model of the crack. However, the affected area may contain complex geometric details (such as the depth variation of the crack) and inaccurate depth measurements caused by humidity and light interference. Directly reconstructing or interpolating the depth of these areas may make it difficult to accurately capture all details and overcome interference, resulting in insufficient accuracy of the reconstructed depth data, affecting the accuracy of the final three-dimensional model of the crack.

[0161] In this regard, the present application further proposes reconstructing the depth value of the affected area according to the geometric constraints of the registered two-dimensional crack profile, and the steps of obtaining the corrected depth data include:

[0162] Divide the impact area into multiple scale levels;

[0163] Apply a depth interpolation algorithm with different resolution parameters for each scale level;

[0164] Starting from the coarsest scale, the depth reconstruction result is refined step by step;

[0165] At each scale level, the depth reconstruction results are adjusted according to the geometric constraints of the registered 2D crack profiles;

[0166] The depth reconstruction results after adjustment at each scale level are fused to generate the corrected depth data.

[0167] Among them, this scheme aims to improve the accuracy of depth reconstruction of the affected area through multi-scale processing and geometric constraint adjustment, thereby obtaining more accurate corrected depth data.

[0168] First, divide the affected area into multiple scale levels. This allows for targeted processing of features of varying sizes and complexity within the affected area, avoiding the potential loss of detail or noise amplification that can occur with single-scale processing. For example, the affected area can be divided into three scale levels: coarse, medium, and fine.

[0169] Next, a depth interpolation algorithm with different resolution parameters is applied to each scale level. This means that the most suitable interpolation algorithm and parameters can be selected based on the characteristics of the current processing scale. For example, lower resolution parameters are used at coarse scales to quickly obtain the overall depth trend, and higher resolution parameters are used at fine scales to finely restore depth details, thereby improving the depth reconstruction accuracy of regions at different scales. For example, bilinear interpolation can be used at coarse scales, bicubic interpolation can be used at medium scales, and spline interpolation can be used at fine scales. The resolution parameter can be reflected in the density of the interpolation grid or the size of the interpolation kernel.

[0170] Then, starting from the coarsest scale, the depth reconstruction is gradually refined. This is a global-to-local processing strategy, first building a coarse depth model and then gradually adding details. This approach helps improve the stability and robustness of the reconstruction process and reduces error accumulation. For example, depth interpolation is first performed at a coarse scale to obtain a preliminary depth map. This depth map is then used as the initial input for mid-scale processing, where more refined interpolation is performed, and so on, until the finest scale.

[0171] At each scale level, the depth reconstruction results are adjusted based on the geometric constraints of the registered 2D fracture profiles. The extracted 2D fracture profiles, which have been registered with the depth data, are used as strong prior information to guide and correct the depth reconstruction results at the current scale. For example, the known 2D position and shape of the fracture can constrain the range or trend of its depth variation, thereby correcting for any deviations that may occur during interpolation or reconstruction, ensuring that the reconstructed depth data is more consistent with the actual fracture geometry, especially in fractured areas. This adjustment can be achieved by introducing weights or penalties based on the fracture profile during the interpolation process, or by performing post-processing corrections on the depth values ​​based on the fracture profile after interpolation is complete.

[0172] By utilizing the two-dimensional crack contour information obtained when identifying the affected area by identifying the humidity change area, the light reflection area, and the location of the aligned two-dimensional crack contour, this scheme can effectively geometrically constrain the depth reconstruction results at each stage of multi-scale processing, thereby overcoming the influence of humidity and light interference on depth measurement and improving the accuracy of depth reconstruction in the affected area.

[0173] Finally, the depth reconstruction results after adjustments at each scale level are fused to generate corrected depth data. The depth information, processed at different scales and geometrically constrained using the 2D fracture profile, is integrated to produce the final, highly accurate corrected depth data that is highly consistent with the 2D fracture profile. This lays the foundation for the subsequent generation of an accurate 3D fracture model. Fusion can be achieved through weighted averaging, multi-resolution analysis, or confidence-based fusion methods.

[0174] In some of the aforementioned solutions of this application, it was proposed to calculate the principal direction distribution of pixels within each segmented region based on the directional information in the polarization angle image. Based on the continuity and consistency of the principal direction distribution, the intersection points of intersecting cracks were identified and separated to extract the two-dimensional crack contours. However, in practical applications, especially in situations where the crack network is complex, the intersection points are dense, or the intersection angles are small, relying solely on the continuity and consistency of the principal direction distribution may make it difficult to accurately identify and separate all intersection points, which can easily lead to misjudgments or omissions, affecting the accuracy of subsequent crack contour extraction.

[0175] In this regard, the present application further proposes to calculate the directional distribution gradient of each pixel point based on the continuity and consistency of the main direction distribution to generate a directional gradient map; set multiple gradient thresholds on the directional gradient map to form multi-level directional gradient regions; apply adaptive adjustment of the angular resolution to each directional gradient region to improve the angular resolution capability of the small-angle intersection region; in the directional gradient region after the angular resolution is adaptively adjusted, detect directional mutation points and mark them as intersection point candidates; cluster the intersection point candidates and merge candidate points whose distance is less than a preset threshold; identify and separate the intersection points of the cross cracks based on the clustered merging results.

[0176] The directional distribution gradient of each pixel is calculated based on the continuity and consistency of the main direction distribution to generate a directional gradient map. This step can effectively highlight areas with dramatic directional changes by quantifying the degree of change in the main direction at the pixel point. These areas often correspond to crack intersections or areas with sharp changes in direction, providing a basis for subsequent intersection detection. For example, the rate of change or second-order derivative of the main direction within a local neighborhood can be calculated as a gradient. As a result, locations with significant directional changes appear as high-value areas on the gradient map.

[0177] Furthermore, multiple gradient thresholds are set on the directional gradient map to form multi-level directional gradient regions. By setting different thresholds, regions with different degrees of directional change can be distinguished, forming multi-level analysis regions from coarse to fine. For example, three gradient thresholds, high, medium, and low, can be set to divide the gradient map into high gradient regions, medium gradient regions, and low gradient regions. This helps capture potential intersection information at different scales. High gradient regions may correspond to obvious intersections, while medium and low gradient regions may contain small-angle intersections or regions with gently changing directions.

[0178] Then, adaptive angular resolution adjustment is applied to each directional gradient region to improve the angular resolution capability of small-angle intersection regions. An adaptive angular resolution adjustment strategy is adopted for different gradient regions. For example, for regions with lower gradient values ​​(which may correspond to small-angle intersections), a higher angular resolution can be used for analysis, so that even small directional changes can be effectively detected. For regions with higher gradient values, a lower angular resolution can be used to improve computational efficiency. This adaptive adjustment enables the solution to analyze directional changes more finely and avoid missed judgments.

[0179] Within the directional gradient region after adaptively adjusting the angular resolution, directional abrupt changes are detected and marked as candidate intersection points. In the refined directional gradient map, directional abrupt changes represent locations where the crack direction changes significantly. These points are preliminarily identified as potential intersection points, forming a candidate set. For example, directional abrupt changes can be identified by searching for local maxima in the adjusted directional gradient region or by using a specific corner detection algorithm.

[0180] Next, the intersection point candidates are clustered, merging those with a distance less than a preset threshold. Because the detected directional change points can be very dense, a true intersection point may correspond to multiple adjacent candidate points. By clustering and merging candidate points with similar distances, redundancy can be effectively removed, consolidating multiple candidate points belonging to the same intersection into a single representative point. For example, a density-based clustering algorithm or a simple distance threshold clustering method can be used. This step improves the accuracy and robustness of intersection point identification.

[0181] Finally, based on the clustering and merging results, the intersection points of the cross cracks are identified and separated. After the above steps, the cluster merging results obtained represent the locations of the cross crack intersection points that are finally identified. Based on these accurately identified intersection points, the intersecting crack segments can be effectively separated, laying an accurate foundation for the subsequent extraction of independent crack contours. Compared with simple judgments based solely on the continuity and consistency of the main direction distribution, the introduction of directional gradient analysis, multi-level thresholds, adaptive adjustment of angular resolution, and cluster merging steps enables this solution to more accurately locate and separate intersection points when dealing with difficult scenarios such as complex crack networks, dense intersections, or small-angle intersections, thereby improving the accuracy of subsequent crack contour extraction.

[0182] In some of the above-mentioned schemes of the present application, it is proposed to cluster the intersection point candidates and merge the candidate points with a distance less than a preset threshold to identify and separate the intersection points of the cross cracks. However, when using a single fixed merging threshold for clustering, it is difficult to adapt to the uneven distribution density of the intersection point candidates, which may lead to the mistaken merging of multiple real intersection points in dense areas, or the inability to effectively merge candidate points belonging to the same real intersection point in sparse areas, affecting the accuracy of intersection point identification.

[0183] In this regard, the present application further proposes clustering the intersection point candidates, and the steps of merging candidate points whose distance is less than a preset threshold include:

[0184] Calculate the local density of the candidate distribution area of ​​the intersection point;

[0185] The candidate distribution area of ​​the intersection point is divided into high-density area and low-density area according to the local density;

[0186] A first merging threshold is set for the high-density area, and a second merging threshold is set for the low-density area, where the second merging threshold is greater than the first merging threshold;

[0187] Adaptively cluster the intersection point candidates according to the first merging threshold and the second merging threshold to merge candidate points whose distance is less than a preset threshold.

[0188] Among them, the local density of the intersection candidate distribution area can be calculated in a variety of ways. For example, the number of other candidate points within a preset radius of each intersection candidate can be calculated, or the average distance of each candidate point to its k nearest neighbor candidate points can be calculated. In this way, a density value reflecting the degree of aggregation of candidate points can be obtained. According to the calculated local density, the entire candidate point distribution area can be divided into different areas. As a preferred embodiment, a density threshold can be set, and the area with a density higher than the threshold is marked as a high-density area, and the area with a density lower than or equal to the threshold is marked as a low-density area. The density threshold can be adjusted according to the actual application scenario and data characteristics.

[0189] Furthermore, different merging thresholds are set for the divided high-density and low-density areas. In high-density areas, the intersection candidate points may be very dense, and may contain multiple real intersection points. In order to avoid mistakenly merging these different real intersection points into one, a relatively small first merging threshold is needed for clustering. In low-density areas, the intersection candidate points are relatively sparse, and the distance between candidate points belonging to the same real intersection point may be slightly larger, and there may be some isolated points that are misdetected. In order to effectively merge candidate points belonging to the same real intersection point and have a certain degree of robustness against misdetected points, a relatively large second merging threshold can be used for clustering. The second merging threshold is set to be larger than the first merging threshold, which reflects the differentiated processing strategy for different density areas.

[0190] This scheme is associated with the steps of identifying and separating the intersection points of cross cracks. After identifying the intersection point candidates, this scheme provides a more refined basis for subsequent clustering by calculating the local density and performing regional division. By adopting a stricter merging standard (smaller threshold) in high-density areas, the error of merging multiple real intersection points can be reduced; by adopting a looser standard (larger threshold) in low-density areas, the success rate of merging candidate points belonging to the same real intersection point can be improved. Therefore, based on the identification of the intersection point candidates, this scheme improves the accuracy of the final identified intersection point by adaptively adjusting the merging threshold, overcoming the limitation that a single fixed threshold is difficult to adapt to different distribution densities. For example, in high-density areas, the first merging threshold can be set to a few pixels, while in low-density areas, the second merging threshold can be set to more than a dozen pixels. The specific value can be calibrated according to the image resolution and crack characteristics.

[0191] Some of the aforementioned solutions in this application propose methods for quantitative analysis of crack geometry parameters and multi-period evolution status assessment based on three-dimensional crack models. This approach overcomes the difficulties traditional methods face in performing accurate three-dimensional measurements and reliable multi-period comparisons in complex outdoor environments. However, even if a relatively accurate single-period three-dimensional crack model is obtained, the challenge remains to effectively utilize multi-period model data for dynamic and quantitative evolution analysis, trend prediction, and risk management, while also adjusting subsequent detection strategies based on the evolutionary state.

[0192] In this regard, the present application further proposes the following steps for quantitative analysis of fracture geometric parameters and multi-period evolution state assessment based on a three-dimensional fracture model:

[0193] Collect three-dimensional fracture model data at multiple time points;

[0194] Calculate the rate of change of crack geometric parameters between adjacent time points;

[0195] Determine the crack evolution speed based on the change rate;

[0196] When the crack evolution rate exceeds the first preset threshold, the next detection cycle is automatically shortened; when the crack evolution rate exceeds the second preset threshold, the real-time monitoring mode is triggered; based on the time series analysis method, multi-period crack data is processed to calculate the crack growth rate and acceleration; the future evolution trend of the crack is predicted based on the crack growth rate and acceleration; and a risk assessment report is generated based on the future evolution trend of the crack.

[0197] Among them, based on the generated high-precision three-dimensional fracture model, this solution further uses multi-period data to conduct dynamic and quantitative evolution analysis, trend prediction and risk assessment, and realizes adaptive adjustment of detection strategies, thus solving the problem of how to reliably evaluate the long-term development status of fractures.

[0198] Specifically, collecting 3D fracture model data at multiple time points provides fundamental data support for subsequent evolution analysis. Based on these multi-period 3D models, the rate of change of fracture geometric parameters between adjacent time points is calculated, enabling the actual development of the fracture to be quantified over different detection cycles. Fracture geometric parameters can include width, depth, length, or volume.

[0199] For example, the change in the maximum width or average depth of the crack between two adjacent detections can be calculated. The crack evolution rate is determined based on the calculated rate of change, providing a quantitative indicator of the current speed of crack development. For example, the evolution rate can be expressed as the number of millimeters that the crack width increases each year. Based on the comparison between the crack evolution rate and the preset threshold, when the crack evolution rate exceeds the first preset threshold, the next detection cycle is automatically shortened, thereby increasing the monitoring frequency of rapidly developing cracks; when the crack evolution rate exceeds the second preset threshold, the real-time monitoring mode is triggered, indicating that the crack may be in a rapid deterioration stage and that higher-level monitoring measures need to be taken immediately. The first preset threshold can be set to, for example, a width growth rate of 0.1 mm per year, and the second preset threshold can be set to, for example, a width growth rate of 0.5 mm per year. This adaptive adjustment of the detection cycle based on the evolution rate improves detection efficiency and the timeliness of risk response.

[0200] Furthermore, by processing multiple periods of fracture data using time series analysis methods and calculating fracture propagation rates and accelerations, we can more comprehensively and stably reveal the long-term evolution patterns and dynamic trends of fractures, avoiding the potential for randomness associated with relying solely on data from two adjacent periods. Time series analysis methods can employ methods such as the Autoregressive Integrated Moving Average (ARIMA) model or the Long Short-Term Memory (LSTM) network. Using the calculated fracture propagation rates and accelerations, we can predict future fracture evolution trends, providing a scientific basis for predicting the long-term state of fracture development.

[0201] Finally, a risk assessment report is generated based on the predicted future evolution trend of cracks, combining quantitative analysis results, trend prediction and structural safety assessment to provide decision support for bridge managers to formulate maintenance, reinforcement or emergency measures.

[0202] Secondly, refer to Figure 2 , this application further proposes an AR-based bridge crack detection and analysis system, including:

[0203] Monitoring module 1, used to monitor the ambient light parameters of the bridge detection area;

[0204] Adjustment module 2, used to adjust the lighting output according to the ambient light parameters to stabilize the imaging lighting conditions of the crack area;

[0205] An acquisition module 3 is used to collect crack images at different polarization angles under imaging illumination conditions in a stable crack region to obtain polarization information of the crack;

[0206] Calculation module 4, used to calculate the Stokes parameters of polarization information and generate polarization degree image and polarization angle image;

[0207] Extraction module 5, used for extracting the two-dimensional contour of the crack based on the polarization degree image and the polarization angle image;

[0208] A generation module 6 is used to fuse the two-dimensional crack profile with the depth data to generate a three-dimensional crack model;

[0209] Evaluation module 7 is used to perform quantitative analysis of fracture geometric parameters and multi-stage evolution state evaluation based on the three-dimensional fracture model.

[0210] By monitoring ambient light and adjusting lighting, using polarization information to enhance contrast, extracting two-dimensional contours and fusing them with depth to generate a three-dimensional model, and performing analysis based on the three-dimensional model, the impact of complex outdoor lighting conditions on image acquisition quality is overcome, the accuracy of crack identification and three-dimensional modeling is improved, and reliable multi-period evolution status assessment is supported.

[0211] In addition, in some preferred embodiments, the AR-based bridge crack detection and analysis system proposed in this application can perform any one of the steps in the above method.

[0212] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, various modifications and variations of the present application are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A bridge crack detection and analysis method based on AR, characterized in that: include: Monitor the ambient light parameters in the bridge inspection area; adjusting the lighting output according to the ambient light parameters to stabilize the imaging lighting conditions of the crack area; Crack images are collected at different polarization angles under stable imaging illumination conditions in the crack area to obtain polarization information of the crack. Calculating the Stokes parameters of the polarization information to generate a degree of polarization image and a polarization angle image; Extracting a two-dimensional contour of the crack based on the polarization degree image and the polarization angle image; fusing the two-dimensional crack profile with the depth data to generate a three-dimensional crack model; Based on the three-dimensional fracture model, quantitative analysis of fracture geometric parameters and multi-stage evolution state evaluation are performed.

2. The bridge crack detection and analysis method based on AR according to claim 1 is characterized in that: The step of collecting crack images at different polarization angles under imaging illumination conditions of a stable crack region to obtain polarization information of the crack comprises: Inertial measurement unit data is recorded synchronously while acquiring each crack image frame at different polarization angles under imaging illumination conditions in a stable crack region; Select a frame of image in the sequence as the reference frame; By extracting and matching visual feature points between each frame image, combined with the inertial measurement unit data, estimating the pose transformation of each frame relative to the reference frame; According to the posture transformation, geometric correction is performed on each frame of the crack image so that it is aligned with the reference frame at the pixel level; The polarization information is calculated using the corrected multi-angle crack images.

3. The bridge crack detection and analysis method based on AR according to claim 1 is characterized in that: The step of extracting the two-dimensional contour of the crack based on the polarization degree image and the polarization angle image comprises: performing image enhancement processing on the polarization degree image and the polarization angle image to improve the contrast between the crack and the background; Perform preliminary segmentation on the image after improving the contrast between cracks and background, and mark potential independent crack areas; Calculating the main direction distribution of pixels in each segmented area based on the direction information in the polarization angle image; identifying and separating intersection points of cross cracks based on the continuity and consistency of the distribution of the principal directions; Extracting the two-dimensional crack profile for each identified independent crack region based on the intersection points of the identified and separated cross cracks; The extracted two-dimensional contours of the cracks are numbered and marked to establish the spatial topological relationship of the cracks.

4. The bridge crack detection and analysis method based on AR according to claim 1 is characterized in that: The step of fusing the two-dimensional crack profile with the depth data to generate a three-dimensional crack model comprises: Place multiple marking points in the area around the crack; Each time a crack image is collected, the three-dimensional coordinate information of the marking point is collected synchronously; Establishing a local coordinate system based on the three-dimensional coordinate information of the marking point; Converting the two-dimensional profile and depth data of the crack into the local coordinate system; Register the converted 2D fracture profile with the depth data; The registered two-dimensional crack profile is fused with the depth data to generate a three-dimensional crack model; The three-dimensional crack model is spatially associated with the bridge BIM model to record the position information of the crack in the bridge structure.

5. The bridge crack detection and analysis method based on AR according to claim 4 is characterized in that: The step of fusing the registered two-dimensional crack profile with the depth data to generate a three-dimensional crack model comprises: Acquire multispectral images covering visible and near-infrared bands; Calculating a bridge surface moisture distribution map and a reflection intensity map based on the multispectral image; identifying a humidity change area and a light reflection area based on the humidity distribution map and the reflection intensity map; Identifying an affected area based on the locations of the humidity change area, the light reflection area, and the registered two-dimensional crack profile; Reconstruct the depth value of the affected area according to the geometric constraints of the registered two-dimensional crack contour to obtain the corrected depth data; The corrected depth data is fused with the registered two-dimensional crack contour to generate a three-dimensional crack model.

6. The bridge crack detection and analysis method based on AR according to claim 5 is characterized in that: The step of reconstructing the depth value of the affected area according to the geometric constraints of the registered two-dimensional crack profile to obtain corrected depth data includes: Divide the impact area into multiple scale levels; Apply a depth interpolation algorithm with different resolution parameters for each scale level; Starting from the coarsest scale, the depth reconstruction result is refined step by step; At each scale level, the depth reconstruction results are adjusted according to the geometric constraints of the registered 2D crack profiles; The depth reconstruction results after adjustment at each scale level are fused to generate the corrected depth data.

7. The bridge crack detection and analysis method based on AR according to claim 3 is characterized in that: The step of identifying and separating intersection points of cross cracks based on the continuity and consistency of the main direction distribution includes: Calculating the directional distribution gradient of each pixel point based on the continuity and consistency of the main direction distribution to generate a directional gradient map; Setting multiple gradient thresholds on the directional gradient map to form a multi-level directional gradient region; Adaptive adjustment of angular resolution is applied to each directional gradient region to improve the angular resolution capability in small-angle intersection areas; In the directional gradient area after the angular resolution is adaptively adjusted, the directional mutation points are detected and marked as intersection candidates; Clustering the intersection point candidates and merging candidate points whose distance is less than a preset threshold; The intersection points of cross cracks are identified and separated based on the merged results after clustering.

8. The bridge crack detection and analysis method based on AR according to claim 7 is characterized in that: The step of clustering the intersection point candidates and merging candidate points whose distance is less than a preset threshold comprises: Calculate the local density of the candidate distribution area of ​​the intersection point; Dividing the candidate distribution area of ​​the intersection point into a high-density area and a low-density area according to the local density; Setting a first merging threshold for the high-density area and a second merging threshold for the low-density area, wherein the second merging threshold is greater than the first merging threshold; Adaptively cluster the intersection point candidates according to the first merging threshold and the second merging threshold to merge candidate points whose distance is less than a preset threshold.

9. The bridge crack detection and analysis method based on AR according to claim 1 is characterized in that: The steps of performing quantitative analysis of fracture geometric parameters and multi-period evolution state evaluation based on the three-dimensional fracture model include: Collect three-dimensional fracture model data at multiple time points; Calculate the rate of change of crack geometric parameters between adjacent time points; determining a crack evolution rate based on the change rate; When the crack evolution speed exceeds a first preset threshold, the next detection cycle is shortened; when the crack evolution speed exceeds a second preset threshold, a real-time monitoring mode is triggered to obtain multiple periods of crack data, where the second preset threshold is greater than the first preset threshold; Based on the time series analysis method, multi-period crack data is processed to quantitatively analyze the crack geometric parameters, thereby calculating the crack growth rate and acceleration; Performing a multi-period evolution state assessment based on the crack growth rate and acceleration, thereby predicting the future evolution trend of the crack; A risk assessment report is generated based on the future evolution trend of the cracks.

10. A bridge crack detection and analysis system based on AR, characterized in that: include: Monitoring module, used to monitor the ambient light parameters in the bridge detection area; an adjustment module, configured to adjust the lighting output according to the ambient light parameters to stabilize the imaging lighting conditions of the crack area; An acquisition module is used to collect crack images at different polarization angles under imaging illumination conditions in a stable crack region to obtain polarization information of the crack; A calculation module, configured to calculate the Stokes parameter of the polarization information and generate a polarization degree image and a polarization angle image; An extraction module, configured to extract a two-dimensional profile of a crack based on the polarization degree image and the polarization angle image; A generation module, configured to fuse the two-dimensional crack profile with the depth data to generate a three-dimensional crack model; The evaluation module is used to perform quantitative analysis of fracture geometric parameters and multi-stage evolution state evaluation based on the three-dimensional fracture model.

Images