A method and system for detecting cracks on concrete surfaces based on the A3C algorithm
By using multimodal data acquisition and hierarchical A3C algorithm to optimize observation decisions, and combining the improved DUSt3R model for multi-view 3D reconstruction, the problems of lighting and texture interference in concrete surface crack detection are solved, achieving high-precision and reliable crack detection and measurement, and improving inspection efficiency and result consistency.
Patent Information
- Application Number
- CN202610613668.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-07
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2046-05-07
AI Technical Summary
Existing methods for detecting cracks on concrete surfaces are susceptible to interference from changes in lighting, shadows, reflections, and textures. They are difficult to distinguish between fine cracks and non-crack linear textures, lack multi-view scale constraints, have poor consistency in measurement results, and have low inspection efficiency.
Multimodal inspection data is collected, combined with attention-based UNet initial screening and hierarchical A3C observation decision, and multi-view three-dimensional reconstruction is performed through an improved DUSt3R model. The true length and width distribution of cracks and the measurement confidence interval are output, realizing adaptive observation action optimization and evidence threshold gating.
It improves detection accuracy and measurement reliability, can assess uncertainty, and improves inspection efficiency. It is suitable for routine inspections of structures such as bridges and tunnels, and provides traceable and quantifiable detection results.
Smart Images

Figure CN122550477B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of concrete surface crack detection technology, and in particular to a concrete surface crack detection method and system based on the A3C algorithm. Background Technology
[0002] With the increasing demand for digital and intelligent inspection of infrastructure operations and maintenance, automated detection and quantitative assessment technologies for concrete surface cracks have received widespread attention. Existing crack detection methods mostly rely on object detection or semantic segmentation networks based on a single RGB image, or employ traditional image algorithms such as threshold segmentation, edge extraction, and morphological processing. In a few scenarios, manual verification is combined to achieve crack location and width estimation. However, in practical engineering applications, the following problems commonly exist: Single-modal imaging is susceptible to interference from changes in illumination, shadows, reflections, stains, and honeycomb textures. It is difficult to distinguish between fine cracks and non-crack linear textures, leading to both false positives and false negatives with unstable confidence levels. The inspection process usually lacks coordinated optimization of imaging parameters, observation angles, and acquisition resources. The reshoot strategy relies on fixed rules or human experience, making it difficult to form a closed-loop decision-making process. This results in insufficient acquisition of key areas or repeated acquisition, leading to low efficiency. Crack width and length are mostly calculated using pixel domain conversion or single-view approximation, which is significantly affected by shooting distance and perspective distortion. There is a lack of multi-view scale constraints and uncertainty expression, resulting in poor consistency of measurement results and difficulty in providing a reliable range.
[0003] Therefore, how to provide a method and system for detecting cracks on concrete surfaces based on the A3C algorithm is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0004] One objective of this invention is to propose a concrete surface crack detection method and system based on the A3C algorithm. This invention fully utilizes multimodal inspection and acquisition, attention-based UNet crack screening, and layered A3C observation decision to achieve adaptive optimization of observation actions for crack candidate areas. Under evidence threshold gating triggering, it calls the improved DUSt3R to perform multi-view three-dimensional reconstruction and scale measurement, outputting the true crack length, true width distribution, and measurement confidence interval. It has the advantages of high detection accuracy, strong measurement reliability, assessable uncertainty, and high inspection efficiency.
[0005] A method for detecting cracks on concrete surfaces based on the A3C algorithm according to an embodiment of the present invention includes the following steps: Step 1: Collect inspection images of concrete surfaces and simultaneously collect thermal infrared data and structured light depth data, record multimodal imaging parameter information, construct a multi-source heterogeneous original inspection dataset, and establish a mapping function between imaging parameters and crack detectability. Step 2: Perform multi-scale preprocessing on the multi-source heterogeneous original inspection dataset, use Attention UNet to perform initial crack screening, obtain a set of crack candidate regions, and generate candidate region fusion features for each crack candidate region; Step 3: Construct a hierarchical A3C architecture, including a high-level policy network and a low-level execution network. The high-level policy network takes the candidate region fusion features, the observation resource status, and the improved information gain features as inputs and outputs an observation decision vector. The low-level execution network executes observation actions based on the observation decision vector. Step 4: Collect high-resolution multimodal data of the target crack candidate region based on the observation decision vector, and fuse and analyze the data to obtain crack mask, crack skeleton, width estimate and uncertainty; Step 5: Calculate the comprehensive evidence score based on crack mask stability, crack skeleton topology consistency, detection confidence, uncertainty, and multimodal consistency. If the comprehensive evidence score is less than the evidence threshold, return to Step 3 to update the observation decision vector. If the multi-view measurement is marked as true or the uncertainty is greater than the uncertainty threshold, proceed to Step 6. Step 6: Obtain the multi-view, multi-modal sequence of the target crack candidate region and input it into the improved DUSt3R model for three-dimensional reconstruction and scale measurement. The improved DUSt3R model introduces a crack ROI dynamic prompting mechanism, a robust alignment mechanism, a scale anchoring fusion mechanism, and an uncertainty propagation mechanism, and outputs the true crack length, true width distribution, and measurement confidence interval. Step 7: Generate a crack detection report, write the observation action sequence and changes in the comprehensive evidence score into the experience pool, and update the hierarchical A3C architecture parameters through the A3C algorithm.
[0006] Optionally, step one specifically includes: Collect inspection images, corresponding thermal infrared data, and structured light depth data of the surface of the inspected concrete components; The inspection images, corresponding thermal infrared data, and structured light depth data are processed for time synchronization and spatial alignment, including writing a unified timestamp for each frame of data, calibrating the intrinsic parameters of the thermal infrared camera and the structured light camera and obtaining the relative extrinsic parameters, and reprojecting the thermal infrared data and structured light depth data onto the inspection image coordinate system to form a pixel-level aligned multimodal frame. For each multimodal frame, multimodal imaging parameter information is recorded. The multimodal imaging parameter information includes the exposure time and gain of the inspection image, the integration time and temperature measurement range of the thermal infrared camera, the projection intensity and stripe coding parameters of the structured light emitter, and the working distance and viewing angle parameters of each sensor. Multimodal frames that are synchronized in time and aligned in space are associated and stored with their corresponding multimodal imaging parameter information to form a multi-source heterogeneous original inspection dataset. Establish a mapping function between imaging parameters and crack detectability: Collect multiple sets of multimodal frames with different combinations of imaging parameters in a preset calibration area, and calculate the crack detectability index for each set of multimodal frames; Each set of multimodal imaging parameter information is paired with the corresponding crack detectability index to form a sample pair. The multimodal imaging parameter information is quantized into discrete levels according to exposure time, gain, integration time, and projection intensity in sequence. A four-dimensional parameter grid table is constructed. The mean value of the crack detectability index is used as the grid score for each grid point, and the variance is recorded as a stability label to obtain the piecewise lookup table form of the mapping function.
[0007] Optionally, step two specifically includes: Multi-scale preprocessing is performed on the multi-source heterogeneous original inspection dataset. Multi-scale preprocessing includes performing pyramid scaling on the inspection images to obtain images at least two scales, and performing denoising, brightness normalization and contrast stretching on each scale image to form a multi-scale inspection image set. A crack screening is performed on a multi-scale inspection image set using an attention-gated UNet, which is a segmentation network that introduces attention gating at the jump connection between the UNet encoder and decoder. The multi-scale inspection image set is input into the attention UNet to obtain the crack probability map at each scale. After scale alignment of the crack probability maps at each scale, the maximum probability is taken according to the pixel position to form a fused crack probability map. A binary mask is obtained by thresholding the fused crack probability map, and a connected component analysis is performed on the binary mask to obtain a set of crack candidate regions. In the inspection image, the boundary complexity is calculated based on the boundary point set, and the spatial geometric features are constructed based on the centerline of the crack candidate region. The local binary mode histogram (LBP) and oriented gradient histogram (HOG) are calculated on the inspection image block to form texture and gradient features. Data blocks corresponding to the crack candidate region are cropped at pixel-level aligned positions in thermal infrared data and structured light depth data to form multimodal fusion features. In the historical inspection data of the same crack candidate region, the regional mean and binary mask area sequence of the fused crack probability map are extracted in time order and the rate of change of the sequence is calculated to form temporal trend features. Spatial geometric features, texture and gradient features, multimodal fusion features and temporal trend features are spliced and normalized to obtain candidate region fusion features, which are then associated with and stored with the crack candidate region set.
[0008] Optionally, step three specifically includes: A hierarchical A3C architecture is constructed, in which a high-level policy network, a high-level value network, and a low-level execution network are set on the same feature backbone. The high-level policy network outputs the observation decision vector, and the high-level value network outputs the state value corresponding to the observed resource state. The candidate observation view set is composed of discrete viewpoints within the reachable pose range of the camera, and the field of view coverage relationship of the current crack candidate region set is associated with each candidate observation view. Constructing improved information gain features: Based on the fusion features of crack candidate regions, construct observation feature vectors for candidate observation perspectives, construct Fisher information matrix features for observation feature vectors, input observation feature vectors into random forest models to obtain perspective gain regression values, calculate the information correlation between the multimodal data blocks corresponding to candidate observation perspectives and the crack initial screening probability map as mutual information features, and concatenate and normalize the Fisher information matrix features, perspective gain regression values and mutual information features to obtain improved information gain features; Construct the observation resource status, which includes the remaining time budget, remaining energy budget, number of observation actions already performed, current imaging parameter information, and the adjustable range of current multimodal imaging parameter information; The candidate region fusion features, observation resource status, and improved information gain features are input into the high-level policy network, and the observation decision vector is output. The observation decision vector is input into the low-level execution network, which converts the target observation viewpoint into motion trajectory control variables, converts the imaging mode selection and imaging parameter control variables into camera and illumination control commands, and executes observation actions for the target crack candidate region.
[0009] Optionally, step four specifically includes: Based on the observation decision vector, the target crack candidate region, target observation angle, imaging mode selection and imaging parameter control quantity are determined. High-resolution inspection images, thermal infrared data and structured light depth data of the target crack candidate region are acquired simultaneously according to the imaging mode selection to form a high-resolution multimodal data frame. Pixel-level alignment is performed on the high-resolution multimodal data frame to obtain aligned high-resolution multimodal data blocks; Perform multi-source data fusion based on adaptive weights on the aligned high-resolution multimodal data blocks to obtain a fused feature map; Pixel-level classification is performed on the fused feature map to obtain a crack probability map, and thresholding is performed to obtain a binary mask. The binary mask is then morphologically refined to obtain the crack mask. Iterative refinement of the binary mask on the crack mask is performed to obtain a single-pixel wide centerline, and the endpoints and bifurcation points are extracted to form a skeleton topology, thus obtaining the crack skeleton. At each skeleton point of the crack skeleton, the skeleton tangent direction is calculated and the normal direction is taken. The intersection point with the crack mask boundary is searched in the normal direction and the pixel distance between the two intersection points is calculated. The pixel distance corresponding to the skeleton point is used as the pixel domain width estimate, and the pixel domain width estimates of all skeleton points are counted to form a width sequence. At least two crack probability maps are obtained by performing at least two inferences on the same target crack candidate region under the same observation decision vector. The variance of the crack probability map is calculated according to the pixel position to form a pixel-level uncertainty map. The uncertainty is obtained by taking the mean value of the pixel-level uncertainty map within the crack mask region.
[0010] Optionally, step five specifically includes: A historical observation set was constructed based on the crack mask and crack skeleton, and the crack mask stability score, crack skeleton topological consistency score, detection confidence and regional uncertainty and multimodal consistency score were calculated. The crack mask stability score, crack skeleton topology consistency score, detection confidence, regional uncertainty and multimodal consistency score are normalized and then weighted and summed according to preset weights to obtain the comprehensive evidence score. When the overall evidence score is less than the evidence threshold, a reshoot trigger message is generated and the process returns to step three. When the multi-view measurement identifier in the observation decision vector is true or the regional uncertainty is greater than the uncertainty threshold, multi-view measurement trigger information is generated.
[0011] Optionally, the improved DUSt3R model includes an image feature encoding module, a cross-view feature matching module, a point map regression module, and a global geometric alignment module. The image feature encoding module inputs the inspection image, thermal infrared data and structured light depth data from each target observation perspective into the modal branch encoder, outputs three modal feature maps, introduces a crack ROI dynamic prompting mechanism, generates dynamic ROIs based on crack mask morphological expansion, and splices the modal data after zero-filling according to the dynamic ROI and inputs them into the fusion encoding layer, outputting a fusion feature map. The cross-view feature matching module performs pixel-level correlation calculation on the fused feature map of any two target observation views to obtain an initial matching score. It constructs a view map by using feature blocks as nodes and view pairs as edges, and uses the initial matching score as edge weights. It aggregates the sum of the product of the features of adjacent nodes and edge weights for each node to obtain an aggregate message. The aggregate message is concatenated with the node's own features and mapped to obtain updated features. The updated features are then used to weight and update the initial matching score to obtain an enhanced matching score. The dot plot regression module multiplies the fused feature map with the enhanced matching score element-wise to obtain the matching guidance feature, and inputs the matching guidance feature into the dot plot regression network to output the three-dimensional point coordinates and point confidence. An uncertainty propagation mechanism is introduced to read the uncertainty quantification result and reproject it onto the dynamic ROI coordinate system. The pixel-level uncertainty is multiplied with the point confidence element-wise to obtain the propagation weight, and the propagation weight is associated with the three-dimensional point coordinates to output a weighted dot plot. The global geometric alignment module determines the cross-view corresponding pixel set based on the enhanced matching score, constructs a weighted residual set, introduces a robust alignment mechanism, performs threshold truncation on the weights, and uses an iterative weighted alignment process to solve the rigid body transformation parameters, outputs an alignment point map, and introduces a scale anchoring fusion mechanism. Within the scale anchoring region, the pixel scale length is calculated with the known physical scale of the reference object, and the three-dimensional coordinates of the alignment point map are scaled using the scale coefficient to obtain an absolute scale point map. Based on the absolute scale point map, the three-dimensional point set corresponding to the crack skeleton is extracted, and the true length distribution and true width distribution of the crack are calculated. Based on the statistical results of the propagation weights on the three-dimensional point set corresponding to the crack skeleton, the measurement confidence interval is output.
[0012] Optionally, step seven specifically includes: Generate a crack detection report and store it in association with the timestamp information of the multi-source heterogeneous original inspection dataset; An experience pool is constructed, which stores the observation decision vector, the observation actions executed by the low-level execution network, the observation resource status, the comprehensive evidence score, the measurement confidence interval, and the observation resource status at the next time step in chronological order. Continuous entries of the same target crack candidate region are associated as an observation action sequence. A reward signal is generated, which is determined by the increment of the comprehensive evidence score and the change in the length of the measurement confidence interval. The A3C algorithm is used to asynchronously update the parameters of the layered A3C architecture.
[0013] A concrete surface crack detection system based on the A3C algorithm according to an embodiment of the present invention includes the following modules: The acquisition and mapping module is used to acquire inspection images, thermal infrared data and structured light depth data, record imaging parameters, construct the original inspection dataset and establish a detectability mapping function. The preprocessing and initial screening module is used to perform multi-scale preprocessing on the dataset, using attention UNet to initially screen and obtain a set of candidate crack regions, and generating candidate region fusion features; The A3C decision module is used to output observation decision vectors and execute observation actions. The high-resolution fusion module is used to collect high-resolution multimodal data of the target candidate region according to the observation decision, and to obtain the crack mask, skeleton, width estimate and uncertainty by fusion and analysis; The evidence gating module is used to calculate the comprehensive evidence score and apply threshold gating. If the score is not met, it will trigger a return to update the decision. If the triggering conditions are met, it will enter the multi-view measurement. The reconstruction and measurement module is used to acquire multi-view, multi-modal sequence inputs for 3D reconstruction and scale measurement of the improved DUSt3R model, and outputs the true length, true width distribution and measurement confidence interval. The report experience module is used to generate crack detection reports, write the observation action sequence and evidence score changes into the experience pool, and update network parameters through A3C.
[0014] The beneficial effects of this invention are: This invention, based on multi-source heterogeneous inspection data and a closed-loop decision-making mechanism, realizes an integrated detection process for concrete surface cracks from "discovery—verification—measurement—update": By simultaneously acquiring inspection images, thermal infrared data, and structured light depth data, and establishing a mapping function between imaging parameters and crack detectability, the detectability and acquisition consistency under complex lighting, shadow reflection, and texture interference conditions are improved; Attention UNet multi-scale initial screening is adopted to generate candidate region fusion features, improving the ability to distinguish between fine cracks and non-crack linear textures; Furthermore, a hierarchical A3C architecture is introduced to output observation decision vectors under observation resource constraints and drive re-shooting and observation actions, realizing closed-loop adaptive acquisition with comprehensive evidence score as the gate, reducing invalid re-shooting and improving evidence stability; When triggering multi-view measurement, the improved DUSt3R combines ROI dynamic prompts, robust alignment, scale anchoring fusion, and uncertainty propagation to obtain the true length, true width distribution, and measurement confidence interval of cracks, thereby significantly improving the scale drift and perspective error caused by pixel domain approximation, and providing traceable, quantifiable, and confidence interval-based detection results for engineering evaluation. This invention is of great significance for the routine inspection of structures such as bridges, tunnels, and factories. It can improve inspection efficiency and standardization while ensuring accuracy and reliability, and support the early detection, quantifiable assessment, and closed-loop operation and maintenance decision-making for potential cracks. Attached Figure Description
[0015] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a concrete surface crack detection method based on the A3C algorithm proposed in this invention; Figure 2 This is a framework diagram of the improved DUSt3R model in the concrete surface crack detection method based on the A3C algorithm proposed in this invention; Figure 3 This is a framework diagram of a concrete surface crack detection system based on the A3C algorithm proposed in this invention. Detailed Implementation
[0016] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0017] refer to Figures 1-2 A method for detecting cracks on concrete surfaces based on the A3C algorithm includes the following steps: Step 1: Collect inspection images of concrete surfaces and simultaneously collect thermal infrared data and structured light depth data, record multimodal imaging parameter information, construct a multi-source heterogeneous original inspection dataset, and establish a mapping function between imaging parameters and crack detectability. Step 2: Perform multi-scale preprocessing on the multi-source heterogeneous original inspection dataset, use Attention UNet to perform initial crack screening, obtain a set of crack candidate regions, and generate candidate region fusion features for each crack candidate region; Step 3: Construct a hierarchical A3C architecture, including a high-level policy network and a low-level execution network. The high-level policy network takes the candidate region fusion features, the observation resource status, and the improved information gain features as inputs and outputs an observation decision vector. The low-level execution network executes observation actions based on the observation decision vector. Step 4: Collect high-resolution multimodal data of the target crack candidate region based on the observation decision vector, and fuse and analyze the data to obtain crack mask, crack skeleton, width estimate and uncertainty; Step 5: Calculate the comprehensive evidence score based on crack mask stability, crack skeleton topology consistency, detection confidence, uncertainty, and multimodal consistency. If the comprehensive evidence score is less than the evidence threshold, return to Step 3 to update the observation decision vector. If the multi-view measurement is marked as true or the uncertainty is greater than the uncertainty threshold, proceed to Step 6. Step 6: Obtain the multi-view, multi-modal sequence of the target crack candidate region and input it into the improved DUSt3R model for 3D reconstruction and scale measurement. The improved DUSt3R model introduces a crack ROI dynamic prompting mechanism, a robust alignment mechanism, a scale anchoring fusion mechanism, and an uncertainty propagation mechanism, and outputs the true crack length, true width distribution, and measurement confidence interval. Step 7: Generate a crack detection report, write the observation action sequence and changes in the comprehensive evidence score into the experience pool, and update the hierarchical A3C architecture parameters through the A3C algorithm.
[0018] In this embodiment, step one specifically includes: The inspection images, corresponding thermal infrared data, and structured light depth data of the surface of the concrete components are collected. The thermal infrared data is the temperature matrix or grayscale matrix output by the thermal infrared camera, and the structured light depth data is the depth matrix obtained by the structured light emitter and the camera through projection encoding stripes and decoding. The inspection images, corresponding thermal infrared data, and structured light depth data are processed for time synchronization and spatial alignment, including writing a unified timestamp for each frame of data, calibrating the intrinsic parameters of the thermal infrared camera and the structured light camera and obtaining the relative extrinsic parameters, and reprojecting the thermal infrared data and structured light depth data onto the inspection image coordinate system to form a pixel-level aligned multimodal frame. For each multimodal frame, multimodal imaging parameter information is recorded. The multimodal imaging parameter information includes the exposure time and gain of the inspection image, the integration time and temperature measurement range of the thermal infrared camera, the projection intensity and stripe coding parameters of the structured light emitter, and the working distance and viewing angle parameters of each sensor. Multimodal frames that are synchronized in time and aligned in space are associated and stored with their corresponding multimodal imaging parameter information to form a multi-source heterogeneous original inspection dataset. Establish a mapping function between imaging parameters and crack detectability: Collect multiple sets of multimodal frames with different combinations of imaging parameters in a preset calibration area, calculate crack detectability index for each set of multimodal frames. The crack detectability index is obtained by normalizing the brightness contrast and gradient magnitude difference between the crack candidate linear region and the neighboring background in the inspection image, the temperature gradient difference of the corresponding region in the thermal infrared data, and the depth mutation amount of the corresponding region in the structured light depth data, and then weighting and summing them according to preset weights. Each set of multimodal imaging parameter information is paired with the corresponding crack detectability index to form a sample pair. The multimodal imaging parameter information is quantized into discrete levels according to exposure time, gain, integration time, and projection intensity in sequence. A four-dimensional parameter grid table is constructed. The mean value of the crack detectability index is used as the grid score for each grid point, and the variance is recorded as a stability label to obtain the piecewise lookup table form of the mapping function.
[0019] In this embodiment, step two specifically includes: Multi-scale preprocessing is performed on the multi-source heterogeneous original inspection dataset. Multi-scale preprocessing includes performing pyramid scaling on the inspection images to obtain images at least two scales, and performing denoising, brightness normalization and contrast stretching on each scale image to form a multi-scale inspection image set. For multi-scale inspection image sets, attention UNet is used to perform initial crack screening. Attention UNet is a segmentation network that introduces attention gating at the jump connection between the UNet encoder and decoder. Attention gating is used to spatially weight and filter the encoded features before fusing them with the decoded features. The multi-scale inspection image set is input into the attention UNet to obtain the crack probability map at each scale. After scale alignment of the crack probability maps at each scale, the maximum probability is taken according to the pixel position to form a fused crack probability map. Threshold segmentation is performed on the fused crack probability map to obtain a binary mask, and connected component analysis is performed on the binary mask to obtain a set of crack candidate regions. The crack candidate regions are the bounding boxes or minimum bounding polygons of the connected components. Image blocks corresponding to crack candidate regions are extracted from the inspection images, and boundary point sets are extracted. Boundary complexity is calculated based on the boundary point sets. Curvature sequences are calculated based on the centerline fitting of the crack candidate regions, and the mean and variance of the curvature sequences are statistically analyzed to form spatial geometric features. Local binary mode histograms (LBP) and oriented gradient histograms (HOG) are calculated on the inspection image blocks to form texture and gradient features. LBP is an operator that generates binary codes by comparing the gray levels of pixel neighborhoods and statistically analyzes the coding frequency. HOG is a feature that bins the gradient direction of the image blocks and statistically analyzes the amplitude of each bin. Data blocks corresponding to crack candidate regions are extracted from thermal infrared data and structured light depth data at pixel-level alignment positions. The correlation coefficient between thermal infrared data blocks and inspection image blocks and the depth statistics of depth data blocks are calculated to form multimodal fusion features. The regional mean and binary mask area sequences of the fused crack probability map are extracted from the historical inspection data of the same crack candidate region in chronological order, and the rate of change of the sequences is calculated to form temporal trend features. Spatial geometric features, texture and gradient features, multimodal fusion features and temporal trend features are spliced and normalized to obtain candidate region fusion features, which are then associated with and stored with the crack candidate region set.
[0020] In this embodiment, step three specifically includes: A hierarchical A3C architecture is constructed, in which a high-level policy network, a high-level value network, and a low-level execution network are set on the same feature backbone. The high-level policy network outputs the observation decision vector, and the high-level value network outputs the state value corresponding to the observed resource state. The candidate observation view set is composed of discrete viewpoints within the reachable pose range of the camera, and the field of view coverage relationship of the current crack candidate region set is associated with each candidate observation view. Constructing improved information gain features: Based on the fusion features of candidate crack regions, construct observation feature vectors for candidate observation perspectives. The observation feature vectors include the field of view coverage ratio, the change in candidate region scale, the change in multimodal imaging parameters, and the uncertainty indicator of candidate region fusion features. Construct Fisher information matrix features for the observation feature vectors. Fisher information matrix features are generated by the statistics of the sensitivity of the observation feature vectors to the candidate region fusion features on the candidate observation perspective. Input the observation feature vectors into a random forest model to obtain the perspective gain regression value. The random forest model is trained using training samples formed by historical observation actions and corresponding changes in comprehensive evidence scores. Calculate the information correlation between the multimodal data blocks corresponding to the candidate observation perspective and the crack initial screening probability map as the mutual information feature. Concatenate and normalize the Fisher information matrix features, perspective gain regression values, and mutual information features to obtain improved information gain features. Construct the observation resource status, which includes the remaining time budget, remaining energy budget, number of observation actions already performed, current imaging parameter information, and the adjustable range of current multimodal imaging parameter information; The candidate region fusion features, observation resource status and improved information gain features are input into the high-level policy network to output the observation decision vector. The observation decision vector includes the target observation viewpoint, the target crack candidate region, the imaging mode selection, the imaging parameter control quantity and the multi-view measurement label. The observation decision vector is input into the low-level execution network, which converts the target observation viewpoint into motion trajectory control variables, converts the imaging mode selection and imaging parameter control variables into camera and illumination control commands, and executes observation actions for the target crack candidate region.
[0021] In this embodiment, step four specifically includes: Based on the observation decision vector, the target crack candidate region, target observation angle, imaging mode selection and imaging parameter control quantity are determined. High-resolution inspection images, thermal infrared data and structured light depth data of the target crack candidate region are acquired simultaneously according to the imaging mode selection to form a high-resolution multimodal data frame. Pixel-level alignment processing is performed on the high-resolution multimodal data frame. Thermal infrared data and structured light depth data are reprojected onto the high-resolution inspection image coordinate system, and denoising and normalization processing is performed to obtain the aligned high-resolution multimodal data block. Perform multi-source data fusion based on adaptive weights on the aligned high-resolution multimodal data blocks: calculate the sharpness index of the inspection image data block, the temperature gradient stability index of the thermal infrared data block, and the effective pixel ratio index of the structured light depth data block respectively, normalize them and use them as the fusion weights of the corresponding modes, and perform weighted fusion according to the fusion weights to obtain the fused feature map. Pixel-level classification is performed on the fused feature map to obtain a crack probability map, and thresholding is performed to obtain a binary mask. The binary mask is then morphologically refined to remove isolated noise points and close the fracture segments to obtain the crack mask. Iterative refinement of the binary mask on the crack mask is performed to obtain a single-pixel wide centerline, and the endpoints and bifurcation points are extracted to form a skeleton topology, thus obtaining the crack skeleton. At each skeleton point of the crack skeleton, the skeleton tangent direction is calculated and the normal direction is taken. The intersection point with the crack mask boundary is searched in the normal direction and the pixel distance between the two intersection points is calculated. The pixel distance corresponding to the skeleton point is used as the pixel domain width estimate, and the pixel domain width estimates of all skeleton points are counted to form a width sequence. At least two crack probability maps are obtained by performing at least two inferences on the same target crack candidate region under the same observation decision vector. The variance of the crack probability map is calculated according to the pixel position to form a pixel-level uncertainty map. The uncertainty is obtained by taking the mean value of the pixel-level uncertainty map within the crack mask region.
[0022] In this embodiment, step five specifically includes: A historical observation set is constructed based on the crack mask and crack skeleton. The historical observation set consists of the crack mask, crack skeleton, detection confidence and regional uncertainty obtained for the same target crack candidate region under different observation decision vectors, and is stored in sequence according to timestamp. Calculate the crack mask stability score: Select the binary crack mask corresponding to the two most recent observations in the historical observation set, align the two binary crack masks at the pixel level, calculate the number of intersection pixels and the number of union pixels, and use the ratio of the number of intersection pixels to the number of union pixels as the crack mask stability score. Calculate the crack skeleton topology consistency score: Represent the crack skeleton corresponding to the two most recent observations as an undirected graph, where the skeleton pixels are nodes and the eight-neighbor connectivity is an edge. Count the number of endpoints, the number of branch points and the number of connected branches in the two skeleton graphs respectively, and calculate the negative of the sum of the absolute differences of the corresponding statistics as the skeleton topology consistency score. At the same time, calculate the skeleton pixel overlap rate in the same coordinate system of the two skeleton graphs as a supplementary score. The detection confidence and regional uncertainty are obtained. The detection confidence is the average pixel probability of the crack probability map within the crack mask region, and the regional uncertainty is the average pixel uncertainty map within the crack mask region. Calculate the multimodal consistency score: Calculate the mean gradient amplitude of the inspection image, the mean absolute value of the temperature gradient of the thermal infrared data, and the mean absolute value of the depth gradient of the structured light depth data within the crack mask area. After normalizing the three, calculate the negative of the sum of the absolute values of the pairwise differences of the three as the multimodal consistency score. The crack mask stability score, crack skeleton topology consistency score, detection confidence, regional uncertainty and multimodal consistency score are normalized and then weighted and summed according to preset weights to obtain the comprehensive evidence score. When the comprehensive evidence score is less than the evidence threshold, a reshoot trigger information is generated and the process returns to step three. The reshoot trigger information includes the scoring item identifier that caused the comprehensive evidence score to drop and the corresponding target crack candidate area. When the multi-view measurement identifier in the observation decision vector is true or the regional uncertainty is greater than the uncertainty threshold, multi-view measurement trigger information is generated. The multi-view measurement trigger information includes the target crack candidate region, the target observation view set, and the corresponding imaging mode selection.
[0023] In this embodiment, the improved DUSt3R model includes an image feature encoding module, a cross-view feature matching module, a point map regression module, and a global geometric alignment module: The image feature encoding module inputs the inspection image, thermal infrared data, and structured light depth data from each target observation perspective into the modal branch encoder. The modal branch encoder is composed of a series of multi-level convolutional layers, normalization layers, and ReLU activation layers, and outputs three modal feature maps. A crack ROI dynamic prompting mechanism is introduced. The crack mask is read and morphological dilation is performed in the target crack candidate region to obtain an extended mask. The intersection of the extended mask and the bounding box of the target crack candidate region is obtained to obtain the dynamic ROI. The dynamic ROI is cropped on each modal data and zero-filling is performed outside the cropping boundary. The three cropped modal feature maps are concatenated in the channel dimension and input into the fusion encoding layer. The fusion encoding layer performs channel compression through one-dimensional convolution and outputs the fusion feature map. The cross-view feature matching module performs pixel-level correlation calculation on the fused feature maps of any two target observation views. The correlation calculation includes normalizing the feature vectors and then performing a dot product to obtain the correlation matrix. The correlation matrix is then subjected to softmax along the candidate matching dimension to obtain the initial matching score. The feature blocks of each target observation view are used as nodes and view pairs are used as edges to construct a view graph. The initial matching score is used as the edge weight. The sum of the product of the features of the adjacent nodes and the edge weights of each node is aggregated to obtain the aggregate message. The aggregate message is concatenated with the node's own features and then mapped through a fully connected layer to obtain the updated features. The updated features are used to weight and update the initial matching score to obtain the enhanced matching score. The dot plot regression module multiplies the fused feature map with the enhanced matching score element-wise to obtain the matching guidance feature, and inputs the matching guidance feature into the dot plot regression network. The dot plot regression network consists of a convolutional backbone and a regression head. The regression head outputs the three-dimensional point coordinates and point confidence for each pixel. The point confidence is obtained by activating the confidence channel with sigmoid. An uncertainty propagation mechanism is introduced to read the uncertainty quantification result and reproject it onto the dynamic ROI coordinate system. The pixel-level uncertainty is multiplied with the point confidence element-wise to obtain the propagation weight, and the propagation weight is associated with the three-dimensional point coordinates to output a weighted dot plot. The global geometric alignment module determines the cross-view corresponding pixel set based on the enhanced matching score, constructs a weighted residual set, where the residual is the difference in 3D point coordinates of the corresponding pixels, and the weights are the propagation weights in the weighted point map and normalized. A robust alignment mechanism is introduced, which performs threshold truncation on the weights and resets weights below the threshold to zero to remove outlier corresponding points. An iterative weighted alignment process is used to solve the rigid body transformation parameters. The iterative weighted alignment process includes weighting the residuals according to the weights, updating the rigid body transformation, applying the rigid body transformation to the multi-view point map, updating the residuals, and repeating the iteration until the convergence condition is met. The aligned point map is output, and a scale is introduced. The scale anchoring fusion mechanism reads the focal length magnification information within the camera from multimodal imaging parameter information and extracts the scale anchoring region. Within the scale anchoring region, the boundary of the reference object is detected and the pixel scale length is calculated. The pixel scale length is compared with the known physical scale of the reference object to calculate the scaling factor. The three-dimensional coordinates of the alignment point map are scaled using the scaling factor to obtain the absolute scale point map. Based on the absolute scale point map, the three-dimensional point set corresponding to the crack skeleton is extracted and the true length distribution and true width distribution of the crack are calculated. The measurement confidence interval is output based on the statistical results of the propagation weight on the three-dimensional point set corresponding to the crack skeleton. The improved DUSt3R model is similar to the original DUSt3R model in that it adopts a four-module serial structure of image feature encoding, cross-view feature matching, point map regression and global geometric alignment. It extracts features from multi-view inputs and establishes corresponding relationships, regresses pixels to 3D point maps and completes unified coordinate system alignment to achieve 3D reconstruction and scale measurement. The improvements lie in embedding a crack scene mechanism within the four modules: a dynamic crack ROI prompt mechanism is introduced in the image feature encoding module, using the dynamic ROI obtained after crack mask dilation for multimodal cropping and fusion encoding; a view map-based matching enhancement mechanism is introduced in the cross-view feature matching module to improve the cross-view correlation of fine cracks; an uncertainty propagation mechanism is introduced in the point map regression module, coupling pixel uncertainty and point confidence into a weighted output point map; and a robust alignment mechanism is introduced in the global geometric alignment module to remove low-weight outlier corresponding points, and a scale anchoring fusion mechanism is introduced to complete absolute scale calibration. The above improvements make the reconstruction process more robust to complex textures, shadows, reflections, and noise, improve the consistency of fine crack matching, reduce dot plot alignment drift, and obtain the true length and true width distribution with confidence intervals, thereby improving measurement consistency and traceability.
[0024] In this embodiment, step seven specifically includes: Generate a crack detection report, which includes the target crack candidate region identifier, target observation view set, crack mask, crack skeleton, crack true length, crack true width distribution and measurement confidence interval, and is stored in association with the timestamp information of the multi-source heterogeneous original inspection dataset. An experience pool is constructed, which stores the observation decision vector, the observation actions executed by the low-level execution network, the observation resource status, the comprehensive evidence score, the measurement confidence interval, and the observation resource status at the next time step in chronological order. Continuous entries of the same target crack candidate region are associated as an observation action sequence. A reward signal is generated, which is determined by the increment of the comprehensive evidence score and the change in the length of the measurement confidence interval. The increment of the comprehensive evidence score is the difference between the current comprehensive evidence score and the previous comprehensive evidence score, and the change in the length of the measurement confidence interval is the difference between the current measurement confidence interval length and the previous measurement confidence interval length. The A3C algorithm is used to asynchronously update the parameters of the hierarchical A3C architecture. The asynchronous update includes reading the observation action sequence and calculating the advantage term by the parallel sampling thread, updating the output parameters of the observation decision vector by the gradient of the high-level policy network based on the advantage term, and updating the state value parameters by the gradient of the value network based on the advantage term.
[0025] refer to Figure 3 A concrete surface crack detection system based on the A3C algorithm includes the following modules: The acquisition and mapping module is used to acquire inspection images, thermal infrared data and structured light depth data, record imaging parameters, construct the original inspection dataset and establish a detectability mapping function. The preprocessing and initial screening module is used to perform multi-scale preprocessing on the dataset, using attention UNet to initially screen and obtain a set of candidate crack regions, and generating candidate region fusion features; The A3C decision module is used to output observation decision vectors and execute observation actions. The high-resolution fusion module is used to collect high-resolution multimodal data of the target candidate region according to the observation decision, and to obtain the crack mask, skeleton, width estimate and uncertainty by fusion and analysis; The evidence gating module is used to calculate the comprehensive evidence score and apply threshold gating. If the score is not met, it will trigger a return to update the decision. If the triggering conditions are met, it will enter the multi-view measurement. The reconstruction and measurement module is used to acquire multi-view, multi-modal sequence inputs for 3D reconstruction and scale measurement of the improved DUSt3R model, and outputs the true length, true width distribution and measurement confidence interval. The report experience module is used to generate crack detection reports, write the observation action sequence and evidence score changes into the experience pool, and update network parameters through A3C.
[0026] Example 1: To verify the feasibility of this invention in practice, it was applied to the annual inspection of the bottom slab and web of a box girder of an elevated expressway in a city. The bridge had been in service for eight years, and the bottom slab showed signs of water seepage and localized honeycomb pitting. During the day, strong sunlight and traffic caused rapidly changing shadows, and traditional single-RGB photography often misidentified template patterns and stain edges as cracks. The inspection equipment used an integrated multimodal acquisition component, including an RGB camera (5472×3648), a thermal infrared camera (640×512, range -20~120℃), and a structured light depth camera (1280×720). All three data streams were triggered by the same clock and written to a unified timestamp. The thermal infrared and structured light data were reprojected online onto the RGB coordinate system to form pixel-level aligned frames. For ease of reproduction, the preset parameters are as follows: RGB exposure 8ms, gain 6dB; thermal infrared integration 12ms; structured light projection intensity 60%; initial screening threshold 0.45; evidence threshold 0.62; uncertainty threshold 0.18; time budget of 180s per 10m², maximum of 3 retakes per area; when triggering multi-view measurement, 3 viewpoints are collected, and the angle between viewpoints is controlled between 15° and 35°.
[0027] During on-site operations, the system first continuously scans and captures images to form a multi-source heterogeneous raw inspection dataset. Then, it collects data with different combinations of exposure, gain, integration time, and projection intensity in a pre-defined calibration area under the bridge. For each data set, brightness contrast, gradient amplitude difference, thermal infrared temperature gradient difference, and depth abrupt change are calculated for both the crack candidate linear region and the background region. These four components are normalized and weighted according to preset weights to obtain a crack detectability index. A four-dimensional parameter grid table is then constructed. During subsequent inspections, when the detectability index of a certain region decreases, the system dynamically adjusts the parameters according to the grid table, selecting a higher score level. Afterward, an attention-based UNet is used to initially screen the multi-scale images to obtain a set of crack candidate regions and candidate region fusion features. The hierarchical A3C outputs an observation decision vector based on the fusion features and observation resource status. The lower-level execution network uses this vector to control the gimbal fine-tuning and re-capture, and sends the new observation results to the evidence gating. If the overall evidence score is insufficient, the process returns to the strategy decision and continues reshooting. When the multi-view measurement markers are true or the uncertainty exceeds the threshold, the improved DUSt3R 3D reconstruction is initiated: In the encoding stage, dynamic ROIs are obtained based on crack mask expansion, and multimodal data is cropped; in the matching stage, view maps are used to enhance cross-view correlation; in the regression stage, pixel uncertainty and point confidence are coupled into propagation weights; in the alignment stage, robust truncation is performed to remove outliers, and scale anchoring is performed using a reference scale, outputting the true length, true width distribution, and measurement confidence intervals. Upon completion of the task, a report is generated and written to the experience pool, and A3C updates asynchronously according to the advantages.
[0028] To demonstrate the effectiveness, the same bridge was divided into six representative inspection areas (each approximately 10m², containing typical interferences such as water seepage shadow areas, honeycomb pitted areas, and dense template texture areas). Three methods were compared: Method A was "single RGB + ordinary UNet segmentation + fixed shooting path + pixel-converted width"; Method B was "single RGB + threshold / edge / morphology + manual reshoot"; and the present invention is "multimodal + attention UNet + layered A3C closed loop + improved DUSt3R measurement". Manual verification used crack micrometers and feeler gauges to randomly check the crack width as the true value. The comprehensive data is shown in Table 1 below. Table 1 Summary of Inspection and Measurement Comparison in the Inspection Area
[0029] As shown in Table 1, the present invention maintains high and stable detection accuracy in typical interference areas: F1 ranges from 0.89 to 0.95, with an average of approximately 0.92; Method A averages approximately 0.83, with increased false detections mainly occurring in areas with dense template textures and rapidly changing shadows; Method B averages approximately 0.72, significantly affected by threshold drift and heavily reliant on the operator's experience in re-shooting. Regarding false detections, the present invention averages approximately 3.2 per 10m², while Method A averages approximately 11.7 per 10m² and Method B approximately 18.7 per 10m², indicating that multimodal consistency and evidence gating effectively suppress "crack-like textures." In terms of measurement, the average width MAE of this invention is approximately 0.055 mm, significantly smaller than that of method A (approximately 0.127 mm) and method B (approximately 0.212 mm). Simultaneously, the average half-width of the confidence interval given by this invention is approximately 0.065 mm, while that of method A is approximately 0.187 mm. This demonstrates that the scale anchoring and uncertainty propagation of the improved DUSt3R can transform the "pixel domain width" into a more stable "true width distribution + confidence interval." Regarding time consumption, the average single-area measurement of this invention is approximately 140 seconds, slightly higher than method A (approximately 125 seconds), but significantly lower than method B (approximately 213 seconds). Considering the reduction in false positives leading to decreased manual review and rework, the overall project duration is more controllable.
[0030] In summary, this invention improves data quality under complex lighting and surface textures through multimodal acquisition and detectability mapping, enhances the detection capability of fine cracks through multi-scale initial screening using Attention UNet, reduces invalid retakes and improves evidence stability through hierarchical A3C closed-loop decision-making, and achieves millimeter-level consistent measurement and confidence interval output using an improved DUSt3R under trigger conditions. Thus, in real bridge inspection, it simultaneously achieves lower false detections, more reliable measurements, and stronger traceability.
[0031] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for detecting cracks on concrete surfaces based on the A3C algorithm, characterized in that, Includes the following steps: Step 1: Collect inspection images of concrete surfaces and simultaneously collect thermal infrared data and structured light depth data, record multimodal imaging parameter information, construct a multi-source heterogeneous original inspection dataset, and establish a mapping function between imaging parameters and crack detectability. Step 2: Perform multi-scale preprocessing on the multi-source heterogeneous original inspection dataset, use Attention UNet to perform initial crack screening, obtain a set of crack candidate regions, and generate candidate region fusion features for each crack candidate region; Step 3: Construct a hierarchical A3C architecture, including a high-level policy network and a low-level execution network. The high-level policy network takes the candidate region fusion features, the observation resource status, and the improved information gain features as inputs and outputs an observation decision vector. The low-level execution network executes observation actions based on the observation decision vector. Step 4: Collect high-resolution multimodal data of the target crack candidate region based on the observation decision vector, and fuse and analyze the data to obtain crack mask, crack skeleton, width estimate and uncertainty; Step 5: Calculate the comprehensive evidence score based on crack mask stability, crack skeleton topology consistency, detection confidence, uncertainty, and multimodal consistency. If the comprehensive evidence score is less than the evidence threshold, return to Step 3 to update the observation decision vector. If the multi-view measurement is marked as true or the uncertainty is greater than the uncertainty threshold, proceed to Step 6. Step 6: Obtain the multi-view, multi-modal sequence of the target crack candidate region and input it into the improved DUSt3R model for three-dimensional reconstruction and scale measurement. The improved DUSt3R model introduces a crack ROI dynamic prompting mechanism, a robust alignment mechanism, a scale anchoring fusion mechanism, and an uncertainty propagation mechanism, and outputs the true crack length, true width distribution, and measurement confidence interval. Step 7: Generate a crack detection report, write the observation action sequence and changes in the comprehensive evidence score into the experience pool, and update the hierarchical A3C architecture parameters through the A3C algorithm.
2. The method for detecting concrete surface cracks based on the A3C algorithm according to claim 1, characterized in that, Step one specifically includes: Collect inspection images, corresponding thermal infrared data, and structured light depth data of the surface of the inspected concrete components; The inspection images, corresponding thermal infrared data, and structured light depth data are processed for time synchronization and spatial alignment, including writing a unified timestamp for each frame of data, calibrating the intrinsic parameters of the thermal infrared camera and the structured light camera and obtaining the relative extrinsic parameters, and reprojecting the thermal infrared data and structured light depth data onto the inspection image coordinate system to form a pixel-level aligned multimodal frame. For each multimodal frame, multimodal imaging parameter information is recorded. The multimodal imaging parameter information includes the exposure time and gain of the inspection image, the integration time and temperature measurement range of the thermal infrared camera, the projection intensity and stripe coding parameters of the structured light emitter, and the working distance and viewing angle parameters of each sensor. Multimodal frames that are synchronized in time and aligned in space are associated and stored with their corresponding multimodal imaging parameter information to form a multi-source heterogeneous original inspection dataset. Establish a mapping function between imaging parameters and crack detectability: Collect multiple sets of multimodal frames with different combinations of imaging parameters in a preset calibration area, and calculate the crack detectability index for each set of multimodal frames; Each set of multimodal imaging parameter information is paired with the corresponding crack detectability index to form a sample pair. The multimodal imaging parameter information is quantized into discrete levels according to exposure time, gain, integration time, and projection intensity in sequence. A four-dimensional parameter grid table is constructed. The mean value of the crack detectability index is used as the grid score for each grid point, and the variance is recorded as a stability label to obtain the piecewise lookup table form of the mapping function.
3. The method for detecting concrete surface cracks based on the A3C algorithm according to claim 1, characterized in that, Step two specifically includes: Multi-scale preprocessing is performed on the multi-source heterogeneous original inspection dataset. Multi-scale preprocessing includes performing pyramid scaling on the inspection images to obtain images at least two scales, and performing denoising, brightness normalization and contrast stretching on each scale image to form a multi-scale inspection image set. A crack screening is performed on a multi-scale inspection image set using an attention-gated UNet, which is a segmentation network that introduces attention gating at the jump connection between the UNet encoder and decoder. The multi-scale inspection image set is input into the attention UNet to obtain the crack probability map at each scale. After scale alignment of the crack probability maps at each scale, the maximum probability is taken according to the pixel position to form a fused crack probability map. A binary mask is obtained by thresholding the fused crack probability map, and a connected component analysis is performed on the binary mask to obtain a set of crack candidate regions. In the inspection image, the boundary complexity is calculated based on the boundary point set, and the spatial geometric features are constructed based on the centerline of the crack candidate region. The local binary mode histogram (LBP) and oriented gradient histogram (HOG) are calculated on the inspection image block to form texture and gradient features. Data blocks corresponding to the crack candidate region are cropped at pixel-level aligned positions in thermal infrared data and structured light depth data to form multimodal fusion features. In the historical inspection data of the same crack candidate region, the regional mean and binary mask area sequence of the fused crack probability map are extracted in time order and the rate of change of the sequence is calculated to form temporal trend features. Spatial geometric features, texture and gradient features, multimodal fusion features and temporal trend features are spliced and normalized to obtain candidate region fusion features, which are then associated with and stored with the crack candidate region set.
4. The method for detecting concrete surface cracks based on the A3C algorithm according to claim 1, characterized in that, Step three specifically includes: A hierarchical A3C architecture is constructed, in which a high-level policy network, a high-level value network, and a low-level execution network are set on the same feature backbone. The high-level policy network outputs the observation decision vector, and the high-level value network outputs the state value corresponding to the observed resource state. The candidate observation view set is composed of discrete viewpoints within the reachable pose range of the camera, and the field of view coverage relationship of the current crack candidate region set is associated with each candidate observation view. Constructing improved information gain features: Based on the fusion features of crack candidate regions, construct observation feature vectors for candidate observation perspectives, construct Fisher information matrix features for observation feature vectors, input observation feature vectors into random forest models to obtain perspective gain regression values, calculate the information correlation between the multimodal data blocks corresponding to candidate observation perspectives and the crack initial screening probability map as mutual information features, and concatenate and normalize the Fisher information matrix features, perspective gain regression values and mutual information features to obtain improved information gain features; Construct the observation resource status, which includes the remaining time budget, remaining energy budget, number of observation actions already performed, current imaging parameter information, and the adjustable range of current multimodal imaging parameter information; The candidate region fusion features, observation resource status, and improved information gain features are input into the high-level policy network, and the observation decision vector is output. The observation decision vector is input into the low-level execution network, which converts the target observation viewpoint into motion trajectory control variables, converts the imaging mode selection and imaging parameter control variables into camera and illumination control commands, and executes observation actions for the target crack candidate region.
5. The method for detecting concrete surface cracks based on the A3C algorithm according to claim 1, characterized in that, Step four specifically includes: Based on the observation decision vector, the target crack candidate region, target observation angle, imaging mode selection and imaging parameter control quantity are determined. High-resolution inspection images, thermal infrared data and structured light depth data of the target crack candidate region are acquired simultaneously according to the imaging mode selection to form a high-resolution multimodal data frame. Pixel-level alignment is performed on the high-resolution multimodal data frame to obtain aligned high-resolution multimodal data blocks; Perform multi-source data fusion based on adaptive weights on the aligned high-resolution multimodal data blocks to obtain a fused feature map; Pixel-level classification is performed on the fused feature map to obtain a crack probability map, and thresholding is performed to obtain a binary mask. The binary mask is then morphologically refined to obtain the crack mask. Iterative refinement of the binary mask on the crack mask is performed to obtain a single-pixel wide centerline, and the endpoints and bifurcation points are extracted to form a skeleton topology, thus obtaining the crack skeleton. At each skeleton point of the crack skeleton, the skeleton tangent direction is calculated and the normal direction is taken. The intersection point with the crack mask boundary is searched in the normal direction and the pixel distance between the two intersection points is calculated. The pixel distance corresponding to the skeleton point is used as the pixel domain width estimate, and the pixel domain width estimates of all skeleton points are counted to form a width sequence. At least two crack probability maps are obtained by performing at least two inferences on the same target crack candidate region under the same observation decision vector. The variance of the crack probability map is calculated according to the pixel position to form a pixel-level uncertainty map. The uncertainty is obtained by taking the mean value of the pixel-level uncertainty map within the crack mask region.
6. The method for detecting concrete surface cracks based on the A3C algorithm according to claim 1, characterized in that, Step five specifically includes: A historical observation set was constructed based on the crack mask and crack skeleton, and the crack mask stability score, crack skeleton topological consistency score, detection confidence and regional uncertainty and multimodal consistency score were calculated. The crack mask stability score, crack skeleton topology consistency score, detection confidence, regional uncertainty and multimodal consistency score are normalized and then weighted and summed according to preset weights to obtain the comprehensive evidence score. When the overall evidence score is less than the evidence threshold, a reshoot trigger message is generated and the process returns to step three. When the multi-view measurement identifier in the observation decision vector is true or the regional uncertainty is greater than the uncertainty threshold, multi-view measurement trigger information is generated.
7. The method for detecting concrete surface cracks based on the A3C algorithm according to claim 1, characterized in that, The improved DUSt3R model includes an image feature encoding module, a cross-view feature matching module, a point map regression module, and a global geometric alignment module. The image feature encoding module inputs the inspection image, thermal infrared data and structured light depth data from each target observation perspective into the modal branch encoder, outputs three modal feature maps, introduces a crack ROI dynamic prompting mechanism, generates dynamic ROIs based on crack mask morphological expansion, and splices the modal data after zero-filling according to the dynamic ROI and inputs them into the fusion encoding layer, outputting a fusion feature map. The cross-view feature matching module performs pixel-level correlation calculation on the fused feature map of any two target observation views to obtain an initial matching score. It constructs a view map by using feature blocks as nodes and view pairs as edges, and uses the initial matching score as edge weights. It aggregates the sum of the product of the features of adjacent nodes and edge weights for each node to obtain an aggregate message. The aggregate message is concatenated with the node's own features and mapped to obtain updated features. The updated features are then used to weight and update the initial matching score to obtain an enhanced matching score. The dot plot regression module multiplies the fused feature map with the enhanced matching score element-wise to obtain the matching guidance feature, and inputs the matching guidance feature into the dot plot regression network to output the three-dimensional point coordinates and point confidence. An uncertainty propagation mechanism is introduced to read the uncertainty quantification result and reproject it onto the dynamic ROI coordinate system. The pixel-level uncertainty is multiplied with the point confidence element-wise to obtain the propagation weight, and the propagation weight is associated with the three-dimensional point coordinates to output a weighted dot plot. The global geometric alignment module determines the cross-view corresponding pixel set based on the enhanced matching score, constructs a weighted residual set, introduces a robust alignment mechanism, performs threshold truncation on the weights, and uses an iterative weighted alignment process to solve the rigid body transformation parameters, outputs an alignment point map, and introduces a scale anchoring fusion mechanism. Within the scale anchoring region, the pixel scale length is calculated with the known physical scale of the reference object, and the three-dimensional coordinates of the alignment point map are scaled using the scale coefficient to obtain an absolute scale point map. Based on the absolute scale point map, the three-dimensional point set corresponding to the crack skeleton is extracted, and the true length distribution and true width distribution of the crack are calculated. Based on the statistical results of the propagation weights on the three-dimensional point set corresponding to the crack skeleton, the measurement confidence interval is output.
8. The method for detecting concrete surface cracks based on the A3C algorithm according to claim 1, characterized in that, Step seven specifically includes: Generate a crack detection report and store it in association with the timestamp information of the multi-source heterogeneous original inspection dataset; An experience pool is constructed, which stores the observation decision vector, the observation actions executed by the low-level execution network, the observation resource status, the comprehensive evidence score, the measurement confidence interval, and the observation resource status at the next time step in chronological order. Continuous entries of the same target crack candidate region are associated as an observation action sequence. A reward signal is generated, which is determined by the increment of the comprehensive evidence score and the change in the length of the measurement confidence interval. The A3C algorithm is used to asynchronously update the parameters of the layered A3C architecture.
9. A concrete surface crack detection system based on the A3C algorithm, comprising the concrete surface crack detection method based on the A3C algorithm as described in any one of claims 1 to 8, characterized in that, Includes the following modules: The acquisition and mapping module is used to acquire inspection images, thermal infrared data and structured light depth data, record imaging parameters, construct the original inspection dataset and establish a detectability mapping function. The preprocessing and initial screening module is used to perform multi-scale preprocessing on the dataset, using attention UNet to initially screen and obtain a set of candidate crack regions, and generating candidate region fusion features; The A3C decision module is used to output observation decision vectors and execute observation actions. The high-resolution fusion module is used to collect high-resolution multimodal data of the target candidate region according to the observation decision, and to obtain the crack mask, skeleton, width estimate and uncertainty by fusion and analysis; The evidence gating module is used to calculate the comprehensive evidence score and apply threshold gating. If the score is not met, it will trigger a return to update the decision. If the triggering conditions are met, it will enter the multi-view measurement. The reconstruction and measurement module is used to acquire multi-view, multi-modal sequence inputs for 3D reconstruction and scale measurement of the improved DUSt3R model, and outputs the true length, true width distribution and measurement confidence interval. The report experience module is used to generate crack detection reports, write the observation action sequence and evidence score changes into the experience pool, and update network parameters through A3C.
Citation Information
Patent Citations
Bridge crack grade evaluation method and system based on reinforcement learning algorithm
CN115511836A
Building prefabricated part quality detection method based on multi-modal vision
CN120563462A
