A crack depth estimation method, system, device and storage medium based on crack visual image, structured feature constraint and physical constraint

Abstract

This invention discloses a crack depth estimation method based on crack visual images, structured feature constraints, and physical constraints. The method involves acquiring crack images, video frames, or multimodal images of the target component; performing preprocessing; extracting crack structured features from the preprocessed crack region; inputting these features into a visual feature extraction network to form a visual depth representation; and outputting crack depth values, crack depth ranges, crack depth levels, or depth-related damage indicators based on the crack structured features, visual depth representation, and at least one fusion or constraint module. This invention enables the integration of crack visual information, geometric information, topological information, and mechanistic priors into a unified depth prediction framework, without being limited to single image regression. This achieves highly robust, scalable, and deployable estimation of crack depths in beams, slabs, columns, walls, bridges, tunnel linings, pavements, dams, soil and rock masses, or other components.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary fields of structural health monitoring, intelligent civil engineering inspection, machine vision, image processing, fracture mechanics, pattern recognition, and artificial intelligence. Specifically, it relates to a method, system, device, and storage medium for estimating crack depth based on crack visual images, structured feature constraints, and physical constraints. More particularly, it relates to a method, system, device, and storage medium that uses crack visual images as the main input, crack segmentation enhancement as the foundation, and structured features and physical surrogate priors as training constraints. The proposed solution can be used for crack depth analysis of flexural members in laboratory settings and can also be extended to bridges, tunnels, existing buildings, underground engineering, pavements, dams, and other components or materials with cracks. Background Technology

[0002] Cracks are one of the most common and visually apparent damage characteristics in structures such as concrete, masonry, metal, rock, and composite materials. Engineering has long focused on the presence, distribution, length, width, number, and temporal evolution of cracks. However, among indicators that more directly reflect the degree of internal damage, load-bearing capacity degradation, and the necessity of repair and reinforcement, crack depth is of greater value. Crack width is a visible geometric quantity at the surface, while crack depth more closely reflects the extent of damage propagation inwards and is often more closely related to durability risk, permeability risk, corrosion risk, and structural safety margin.

[0003] Existing technologies can generally be categorized into several types. The first type includes manual or semi-manual measurement methods, such as crack gauges, feeler gauges, crack microscopes, probe-type measuring tools, and cutting profiles. These methods can obtain relatively accurate width or depth at local points, but they are inefficient, highly destructive, and lack representativeness, making them unsuitable for large-scale inspections. The second type includes traditional non-destructive testing methods, such as ultrasonic transmission, shock echo, ground-penetrating radar, and thermal imaging inversion. These methods can detect subsurface damage, but they are costly, sensitive to testing conditions, rely on experience for interpretation, and often require specific working conditions. The third type includes crack detection, segmentation, and width measurement methods based on image processing or deep learning. These methods can effectively address the questions of "whether there is a crack," "where the crack is," and "what the crack surface morphology is like," but most remain at the level of detection, segmentation, and width quantification.

[0004] In terms of crack depth estimation, common approaches in existing publicly available technologies include: directly inputting crack images into convolutional neural networks to regress depth; using thermal imaging, infrared images, or multimodal images in conjunction with machine learning models for depth classification or regression; feeding image features into traditional regressors; or using some kind of neural network or MLP to optimize component crack images for depth prediction. While these approaches expand the boundaries of visual detection, they generally have several limitations: First, most approaches directly map crack images to depth results, failing to fully utilize highly interpretable structural information such as crack geometry, topology, and skeleton. Second, most approaches place all the learning pressure on the main image network, lacking explicit constraints that can be associated with the crack depth mechanism. Third, many publicly available approaches rely on a single model output during both the training and inference phases, lacking robust mechanisms such as multi-level fusion, monotonic ranking constraints, and physical feasible region constraints.

[0005] The applicant also noted that among the existing technologies searched or compared, at least the following representative approaches exist: one is "concrete crack depth prediction model based on deep learning," which focuses on directly inferring depth values ​​from images and pays more attention to single-network image regression; another is "image processing method and system for beam crack depth based on MLP optimization," whose core still leans towards single component scenes and single-network inference logic; yet another approach targets crack segmentation or width measurement optimization, which, although it can output crack regions, main directions, or local maximum width positions, does not establish a unified learning loop between crack surface geometry and internal depth; in addition, there are integrated approaches in academic literature that input crack images into CNNs and then connect them to RF or XGBoost for regression, whose approach still uses images as the sole or primary feature source, lacking structured feature constraints and physical proxy monotonicity constraints. The common problem with the above technical approaches is that although depth prediction has been proposed, the highly interpretable information such as crack morphology, topology, evolution, and mechanism has not been fully organized into a unified technical solution that is interpretable, portable, and deployable.

[0006] Fracture mechanics further shows that crack depth is not an isolated value, but is coupled with crack opening displacement, crack surface length, crack propagation region, material nonlinear damage zone, energy release rate, stress intensity factor, double K fracture criterion, component geometry and boundary stress conditions.

[0007] In engineering vision scenarios, while it is difficult to accurately recover all internal mechanistic quantities directly from ordinary surface images, completely ignoring prior mechanistic mechanisms would also weaken the model's generalization ability. Therefore, establishing a scheme that takes into account the main vision task, the structured geometric description of cracks, crack mechanism surrogate terms, constraint learning, and deployable output would be more conducive to constructing a novel patent solution that is clearly distinguishable from existing technologies. Summary of the Invention

[0008] The purpose of this invention is to provide a crack depth estimation scheme that is not limited to a single image regression, a single table regression, or a single crack segmentation result, so that visual features, structured features, physical surrogate priors, stage information, and feasible region constraints can all serve crack depth prediction.

[0009] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0010] A crack depth estimation method based on crack visual images, structured feature constraints, and physical constraints includes the following steps:

[0011] S1. Acquire crack images, video frames, or multimodal images of the target component;

[0012] S2. Perform one or more preprocessing steps on the crack image, video frame or multimodal image, including crack detection, crack segmentation, crack region enhancement, boundary reconstruction, skeleton extraction or noise suppression.

[0013] S3. Extract the structured features of the cracks from the pre-processed crack region;

[0014] S4. Input crack images, video frames, or multimodal images into a visual feature extraction network to form a visual depth representation;

[0015] S5. Based on the crack structured features, visual depth representation, and at least one fusion or constraint module, output the crack depth value, crack depth range, crack depth level, or depth-related damage index.

[0016] Furthermore, the extracted crack structured features in step S3 include any one or any combination of several of the following: average width, median width, minimum width, maximum width, equivalent average width, quantile width, width dispersion, width coefficient of variation, width skewness, width kurtosis, crack area, crack perimeter, skeleton length, number of endpoints, number of bifurcation points, endpoint spacing, tortuosity, branch density, endpoint density, bounding box width, bounding box height, aspect ratio, convex hull area, solidity, compactness, area-to-perimeter ratio, principal orientation angle, major axis dispersion, minor axis dispersion, elongation, and number of connected components.

[0017] Furthermore, the visual feature extraction network in step S4 is any one or any combination of several of the following: convolutional neural network, residual network, dense connection network, MobileNet-like network, EfficientNet-like network, RepVGG-like network, ConvNeXt-like network, VisionTransformer, Swin Transformer, hybrid convolutional-transformer network, state-space visual network, image pyramid network, and temporal visual network.

[0018] Furthermore, the fusion or constraint module in step S5 can be any one of the following: decision-level fusion module, feature-level fusion module, structured feature constraint module, and physical proxy constraint module.

[0019] Furthermore, the fusion or constraint module is a decision-level fusion module, which uses the first depth prediction result output by the structured feature branch and the second depth prediction result output by the image branch to form meta-features, and then obtains the final depth output through linear regression, ridge regression, elastic network, tree model, gating network or hybrid expert network.

[0020] Furthermore, the fusion or constraint module is a feature-level fusion module, which concatenates, weights, sums, gates, modulates, performs cross-attention interaction, performs bilinear mapping, tensor fusion, graph convolution coupling, or multi-scale alignment of visual depth representation and structured feature encoding representation, and then obtains the final depth output through the fusion regression head.

[0021] Furthermore, the fusion or constraint module includes a structured feature constraint module, which sets up an image master prediction head and a structure auxiliary prediction head, using the image master prediction head as the final deployment output, and applying auxiliary regression loss and consistency loss during training using the structure auxiliary prediction head; the consistency loss is L1 loss, L2 loss, Huber loss, KL divergence loss, distillation loss, cosine similarity loss, ranking consistency loss or a combination thereof.

[0022] Furthermore, the fusion or constraint module includes a physical proxy constraint module. This module employs at least one of the following: crack opening proxy quantity, crack propagation proxy quantity, and crack complexity proxy quantity, to construct a physical-aided prediction head, monotonic ranking constraint, depth range constraint, or mechanism consistency constraint. The crack opening proxy quantity includes any one or more of the following: equivalent average width, upper quantile width, extreme value width, width interval, and width quantile ratio. The crack propagation proxy quantity includes any one or more of the following: crack area, skeleton length, bounding box scale, and propagation area ratio. The crack complexity proxy quantity includes any one or more of the following: tortuosity, bifurcation density, endpoint density, number of connected components, shape elongation, and directional dispersion. The monotonic ranking constraint constructs sample pairs based on the physical proxy comprehensive score, ensuring that the prediction depth of crack samples with higher physical proxy comprehensive scores is not less than the prediction depth of crack samples with lower physical proxy comprehensive scores, allowing for the setting of interval boundaries. The depth range constraint, based on component thickness, component cross-sectional height, protective layer thickness, detection path length, material layering boundaries, historical detection records, or engineering priors, constrains the prediction depth to be within a physically feasible range.

[0023] Furthermore, the method also includes determining the stage of crack evolution, specifically:

[0024] Based on the relationship between the main crack and the secondary crack, the crack end extension state, the width-length coupling state, the bifurcation morphology, the abrupt change in direction, or the load history label, the crack stage category is output, and the crack stage category is introduced into the depth estimation process as additional supervision information, prior constraint information, or gating information.

[0025] Furthermore, the load history label is obtained from damage measurement, cutting profile measurement, crack depth gauge, ultrasonic transmission, shock echo, radar detection, industrial CT, 3D reconstruction, digital image correlation, thermal imaging inversion, manual experience grading, or any combination thereof.

[0026] Furthermore, the method also includes uncertainty assessment, specifically:

[0027] Prediction confidence, confidence intervals, or rejection flags can be generated through deep integration, Dropout, quantile regression, Bayesian regression head, error prediction head, confidence calibration layer, or distributed regression.

[0028] Furthermore, the method also includes:

[0029] Based on fracture mechanics surrogate modeling, a mapping relationship is established between any one or more of the following: stress intensity factor surrogate, double K fracture criterion surrogate, energy release rate surrogate, crack surface opening displacement surrogate, equivalent crack bandwidth surrogate, structural size correction factor, load location factor, support boundary factor, or material brittleness factor, and the mapping relationship is established with crack structure characteristics. This mapping relationship is then used to constrain or correct the final depth prediction.

[0030] This invention provides a crack depth estimation system based on crack visual images, structured feature constraints, and physical constraints, characterized in that it includes:

[0031] The data acquisition module is used to acquire crack images or video frames of the target component;

[0032] The crack segmentation and enhancement module is used for preprocessing crack images;

[0033] The structured feature extraction module is used to extract the structured features of cracks;

[0034] The visual depth representation module is used to extract visual depth representations from crack images;

[0035] A multi-level fusion or constraint module is used to fuse the structured features and visual depth representation and apply constraints;

[0036] The depth output module is used to output crack depth values, ranges, grades, or damage indicators.

[0037] Each module is configured to execute the methods described above.

[0038] The present invention also provides an electronic device, including a processor, a memory, and a program stored in the memory and executable on the processor; when the program is executed by the processor, it implements the method according to any one of claims 1 to 7.

[0039] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any one of claims 1 to 7.

[0040] I. Compared to the technical approach of directly regressing depth from images, this invention does not simply input the crack image into a deep network and directly output the depth value. Instead, it forms a closed-loop structure of "segmentation enhancement—structured features—visual representation—multi-level fusion—constraint learning—final deployment and output". In different implementations, some or all of these modules can be selected according to application requirements. Accordingly, this invention differs from existing image-based direct regression technologies in at least the following aspects: First, it introduces a segmentation enhancement layer, making the visual input more focused on the crack region; second, it introduces a structured feature set, allowing crack width, area, skeleton, and topological information to participate in learning; third, it introduces a constraint learning layer, so that the training of the image master head no longer completely depends on black-box visual representation.

[0041] Second, compared to existing technologies such as MLP-based methods and systems for processing beam crack depth, this invention is not limited to a single component type, a single network form, or a single scenario. Its applicability covers multiple objects including beams, slabs, columns, walls, bridges, tunnels, pavements, and dams, and also encompasses various alternative implementations such as convolutional networks, Transformers, tree models, table encoders, and hybrid expert networks. More importantly, this invention organizes crack segmentation, structured features, visual master head, consistency constraints, physical proxy ranking constraints, and feasible domain range constraints into a unified scheme, rather than optimizing a single regressor.

[0042] Third, compared to existing technologies that only target crack detection, crack segmentation, or crack width measurement, the focus of this invention is not on "identifying cracks" itself, but on establishing an estimation mechanism from surface visual information to internal depth indicators after cracks are detected or segmented. This invention uses crack width, crack propagation area, skeleton length, topological complexity, and crack evolution state as organizeable, constrainable, and fusionable intermediate knowledge, making crack depth estimation the primary objective.

[0043] Fourth, compared to the integrated techniques in academic literature that involve "image feature extraction via CNN followed by RF / XGBoost regression," this invention not only covers decision-level fusion but also further discloses feature-level fusion, structural constraints on the image master head, physical proxy constraints on the image master head, dynamic consistency constraints, ranking constraints, and range constraints, thus forming multiple interconnected yet independently implementable technical paths. Therefore, this invention differs from existing solutions that only employ a single fusion method in terms of module combination, training mechanism, and deployment method.

[0044] Fifth, this invention combines algorithms with engineering objects, data acquisition, crack segmentation, feature extraction, image enhancement, component feasible region constraints, reinforcement suggestion output, and inspection system deployment to form a complete technical solution for crack depth estimation of solid structures, rather than an isolated abstract mathematical process. Compared with technical approaches that only predict depth from images, this invention emphasizes a combination of multi-level fusion, multi-source constraints, master-slave deployment, and mechanism proxy.

[0045] Therefore, compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. Improve the characterization of fine cracks, bifurcated cracks, and cracks in complex backgrounds by segmentation enhancement and structured feature extraction;

[0047] 2. By fusing decision-level and feature-level dual paths, image and structural information can be used collaboratively in both the result layer and the representation layer;

[0048] 3. By using a structural constraint scheme, the final inference stage can still rely solely on the image master head, balancing the richness of training information with the simplicity of deployment;

[0049] 4. By using a physical proxy constraint scheme, the prediction results have a stronger mechanistic explanation space;

[0050] 5. By using sorting constraints, range constraints, and stage constraints, the model has stronger stability, technical integrity, and application adaptability.

[0051] 6. Through open backbone replacement and hot-swappable module design, this invention is not limited by a specific network name, number of parameters or training set size. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0053] Figure 2 This is a detailed flowchart of the present invention;

[0054] Figure 3 This is a schematic diagram of the multi-level fusion and constraint enhancement framework in this invention;

[0055] Figure 4 This is a schematic diagram illustrating the coupling relationship between physical proxy quantity and crack depth in this invention;

[0056] Figure 5 A schematic comparison of the overall effects of different technical approaches;

[0057] Figure 6 A schematic diagram of a typical deployment scenario that can be covered by this invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the application will be further described in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in this invention. All non-innovative embodiments based on this invention by other researchers in the art are within the protection scope of this invention.

[0059] like Figure 1 , 2 As shown in Figures 3 and 4, this invention provides a crack depth estimation method based on crack visual images, structured feature constraints, and physical constraints, comprising the following steps:

[0060] S1. Acquire crack images, video frames, or multimodal images of the target component;

[0061] S2. Perform one or more preprocessing steps on the crack image, video frame or multimodal image, including crack detection, crack segmentation, crack region enhancement, boundary reconstruction, skeleton extraction or noise suppression.

[0062] S3. Extract the structured features of the cracks from the pre-processed crack region;

[0063] S4. Input crack images, video frames, or multimodal images into a visual feature extraction network to form a visual depth representation;

[0064] S5. Based on the crack structured features, visual depth representation, and at least one fusion or constraint module, output the crack depth value, crack depth range, crack depth level, or depth-related damage index.

[0065] Based on the above, the specific steps are as follows:

[0066] 1. Data Acquisition

[0067] The data in this invention originates from laboratory component loading tests. Monotonic loading, cyclic loading, fatigue loading, temperature-load coupled loading, or corrosion-load coupled loading can be applied to beams, slabs, walls, joints, or other specimens, acquiring surface crack images at different stages of stable and unstable crack propagation. These images can be acquired using industrial cameras, SLR cameras, mobile terminals, microscopic imaging devices, video acquisition devices, or drones. During acquisition, the loading stage, component dimensions, ambient lighting, shooting distance, lens focal length, shooting angle, and timestamp can be recorded for subsequent modeling, domain adaptation, or quality control.

[0068] Data can also be obtained from inspections of existing bridges, tunnels, building walls, underground structures, or road surfaces. Crack images can be acquired through handheld terminals, vehicle-mounted terminals, robots, rail platforms, or drones, and weakly supervised labels can be constructed by combining inspection logs, component numbers, damage levels, damage locations, crack length ranges, and known maintenance records.

[0069] The data is not limited to a single visible light image; it can also include thermal infrared images, polarization images, multispectral images, nighttime illumination images, 3D point cloud projection images, or keyframes after video stabilization. Multi-source images can be concatenated in parallel at the input layer, or they can be first converted into a unified crack characterization map before being input into the network.

[0070] 2. Tag Construction and Annotation

[0071] Crack depth labels are obtained through cutting, cross-sectional observation, and manual measurement of crack locations on components. For laboratory components, a cross-section can be taken at the crack location to measure the maximum depth, average depth, or layer depth of the crack propagation. For actual engineering components, monitoring labels can be obtained using crack depth gauges, ultrasound, shock echo, radar, industrial CT, X-ray, endoscopy, or manual experience grading.

[0072] Pixel-level annotations of crack areas are first completed manually on an annotation platform, and then mask labels are exported. Annotation formats can include polygons, free curves, brush regions, pixel-level masks, boundary lines, or the main crack skeleton. To improve robustness, annotations can be either consistent single annotations, annotations reviewed by multiple people, voting annotations, or soft annotations for uncertain areas.

[0073] Furthermore, depth labels do not have to be single absolute values; they can also be interval labels, grade labels, relative ordering labels, crack stage labels, or combinations of multiple labels. For example, cracks can be divided into shallow, intermediate, and deep layers; stage labels can be set according to the pre-, intermediate, and post-failure stages; and relative depth-to-shallow order pairs can be established based on multiple detection records. In this way, even if absolute depth measurement is difficult, supervisory information that can be used for ordering constraints or stage constraints can still be constructed.

[0074] 3. Crack segmentation and input enhancement

[0075] First, a semantic segmentation model for cracks is trained to extract crack region masks from the original image. The segmentation model can be U-Net, DeepLab, SegFormer, Mask2Former, or other networks. To accommodate fine cracks, hole filling, connected component filtering, boundary smoothing, skeletonization, and primary crack-secondary crack separation can also be performed after the segmentation output.

[0076] The segmentation results are used to guide image enhancement. Specifically, this includes: cropping the crack region using a mask and expanding the boundary context; assigning higher attention weights to the crack region; blurring or suppressing the background region; rotating and aligning along the main direction of the crack; and constructing a dual-input system of a magnified crack image and the original image.

[0077] Furthermore, segmentation masks can be used without directly participating in inference, but only as teacher information to guide the learning of the image master head during the training phase. This allows the invention to still only require the original image as input during the deployment phase.

[0078] 4. Structured Feature Extraction

[0079] After binarizing and cleaning the connected components of the crack mask, distance transformation and skeleton extraction are performed, and the local width is approximated by the double distance of the skeleton points. Then, statistics such as average width, median width, extreme width, quantile width, width interval, width variation coefficient, skewness and kurtosis are calculated.

[0080] Based on the crack boundary and skeleton, further calculations are performed on the crack area, perimeter, skeleton length, equivalent average width, number of endpoints, number of bifurcation points, distance between farthest endpoints, tortuosity, branch density, endpoint density, orientation angle, major and minor axis dispersion, elongation, bounding box width and height, convex hull area, solidity, compactness, area-to-perimeter ratio, and region coverage. The following are examples of formula definitions:

[0081] The equivalent average width:

[0082] ;

[0083] in, The area of ​​the crack region. The length of the crack skeleton.

[0084] The tortuosity of the crack satisfies:

[0085] ;

[0086] in, Let be the Euclidean distance between the main endpoints of the crack.

[0087] The crack branch density satisfies:

[0088] ;

[0089] in, This represents the number of crack branch points.

[0090] Density at the crack endpoints:

[0091] ;

[0092] in, This represents the number of crack endpoints.

[0093] Structured features can be grouped by function: opening group, expansion group, topology group, morphology group, stage group, and mechanism surrogate group. Different groups can be input into different encoders, or combined by weights to form surrogate quantities. This facilitates the selection of specific feature groups for different scenarios.

[0094] 5. Visual Depth Representation

[0095] A convolutional neural network backbone is used to extract visual representations, and the final classification layer is replaced with a regression head to output the crack depth. An input method that preserves the crack geometry is preferred, rather than forcibly stretching all images to equilateral squares.

[0096] Alternatively, a Transformer-based backbone can be used to extract visual representations and self-attention can be used to model long-distance dependencies of cracks, thereby better describing long cracks, tortuous cracks, and multi-branched cracks.

[0097] Simultaneously, a dual-path visual backbone can be constructed: one path processes the entire image, and the other path processes the local crack image after segmentation guidance; the two paths of features are then input into the image header after gating, stitching, summing, or cross-attention fusion.

[0098] 6. Decision-level integration

[0099] The structured feature branch first outputs a first depth prediction value from a tree model, linear model, kernel model, tabular neural network or a combination thereof; the image branch outputs a second depth prediction value from the visual backbone; then, two or more prediction values ​​are used as meta-features and input into the fusion unit to generate the final depth estimate.

[0100] The fusion unit can be a linear weighted amplifier, ridge regression, elastic network, extreme random tree, small perceptron, gated network, or hybrid expert network. The fusion unit can be trained using a separate validation set or by using out-of-sample predictions generated through cross-validation as the meta-training set.

[0101] The fusion unit receives not only two depth predictions, but also the confidence scores of each branch, the uncertainties of each branch, the image quality scores, the segmentation quality scores, and the component scene labels, thereby forming a conditional fusion.

[0102] 7. Feature-level fusion

[0103] The visual backbone first outputs visual embeddings, while the structured features, after preprocessing and encoding, output structural embeddings. These two embeddings are then jointly encoded in the representation layer, and finally, the fusion regression head outputs the depth information. To ensure that both embeddings retain usable information, image-assisted heads and structure-assisted heads can be set, forming a joint training structure with a main head and two auxiliary heads.

[0104] Gated fusion, cross-attention fusion, bilinear pooling, graph network coupling, or conditional normalization can also be used to modulate visual features with structured features. For example, the width surrogate in structured features can modulate visual channel weights, and the topological complexity in structured features can modulate multi-scale feature fusion weights.

[0105] Feature-level fusion can also be combined with a distillation mechanism: using the fusion head as the teacher and the image master head as the student, so that only the image master head is retained in the inference stage.

[0106] 8. Structured Feature Constraints

[0107] The image master head serves as the final deployment output, while the structured auxiliary head participates in supervision during training but does not participate in the final deployment. The image master head and the structured auxiliary head each output depth estimates, which are then optimized using the main loss, auxiliary loss, and consistency loss. This avoids the final deployment depending on the structured input, while simultaneously leveraging structural information to improve training stability.

[0108] In the implementation of the image master head with structured feature constraints, the following formula can be further adopted:

[0109] The total loss function satisfies:

[0110] ;

[0111] For the deep regression loss of the main regression branch of the image;

[0112] The auxiliary loss is used for the branch constrained by the physical prior parameters; The consistency constraint loss is used between the image master regression output and the physical proxy constraint output. The ranking constraint loss is constructed based on physical agent scoring; Physical range constraint loss for predicting depth;

[0113] to This is the loss weighting coefficient.

[0114] Consistency loss can be L1, L2, Huber, cosine, or distillation loss; dynamic consistency weight scheduling can also be introduced, for example, weaker in the early stage of training and stronger in the later stage, to avoid prematurely aligning two branches that have not yet been learned.

[0115] The consistency constraint loss satisfies:

[0116] ;

[0117] in, The output of the main regression branch for the image. The output is a branch constrained by the physical prior parameters.

[0118] It can also perform sample-level weighting of consistency constraints based on the credibility of the structural auxiliary head, the size of the residual, the image quality score, or the stage label.

[0119] 9. Physical proxy constraints

[0120] From the structured features, a set of surrogate quantities with more physical meaning is selected, such as opening surrogate quantities, expansion surrogate quantities, and complexity surrogate quantities. The opening surrogate quantity reflects the degree of opening of the crack surface, the expansion surrogate quantity reflects the scale of crack surface expansion, and the complexity surrogate quantity reflects the bends, bifurcations, and damage evolution complexity of the crack path.

[0121] The surrogate quantities are interpolated, standardized, and weighted to form a comprehensive physical surrogate score. The weights of each surrogate quantity can be obtained based on experience, fracture mechanics trends, or training. Then, a physical auxiliary head is used to perform depth-assisted regression, and a sample ranking constraint is constructed based on the comprehensive surrogate score, causing crack samples with higher surrogate scores to tend to have greater predicted depths. This is based on the physical surrogate scoring index. Construct a sample pair ranking relationship. For two crack samples from the same specimen, the same working condition, or the same sample group... If satisfied Then the constraint model output satisfies .

[0122] This ranking constraint ensures that the output of the image's main regression branch conforms to the pattern that the degree of crack opening and the complexity of propagation increase with depth. A physical proxy scoring metric is then constructed. The physical proxy scoring index is used to characterize the overall damage level of the crack and satisfies the following:

[0123] ;

[0124] in, , represents the normalized physical proxy quantity. to These are the weighting coefficients.

[0125] To satisfy The ranking constraint loss is constructed for the sample pairs (i, j):

[0126] ;

[0127] in, For sorting interval, This represents the number of sample pairs.

[0128] Physical proxy constraints also include feasible domain range constraints, which use component thickness, protective layer thickness, component cross-sectional height, detection path length, empirical upper limit or design drawing information to limit depth prediction to not exceeding the geometric upper limit and not lower than the physical lower limit.

[0129] This can be further expressed using the following independent formula:

[0130] When the height of the component is known At that time, a range-constrained loss is constructed from the output of the main regression branch of the image:

[0131] .

[0132] 10. Fracture mechanics integration

[0133] The width, length, bifurcation, and propagation extent of cracks in the surface image are mapped to surrogate terms for stress intensity factors, energy release rates, or double-K fracture criteria. For example, an equivalent crack bandwidth can be constructed based on the crack opening and propagation extent, thus forming a depth-dependent surrogate function.

[0134] Linear elastic fracture mechanics (LEFM) is introduced into monocular depth estimation pipelines. Based on LEFM, for pipelines containing depths of... (referred to as in this article) For structural members with cracks, the stress intensity factor (SIF) at the tip can be expressed as:

[0135] in, For tensile stress For component thickness This is the geometric correction factor.

[0136] On visually measurable surfaces, the most important mechanical response is crack mouth opening displacement. In this invention, the corresponding visual extraction feature is the equivalent average width. The relationship between CMOD and depth is derived as follows:

[0137] ;

[0138] in For elastic modulus, It is the shape function of the opening displacement.

[0139] Therefore, it can be concluded that in local material properties and boundary stress Under relatively consistent assumptions, depth Equivalent width of surface A strong positive correlation exists. However, the propagation of cracks in three-dimensional space is often caused by a combination of Type I (opening) and Type II (slipping) forces, and its propagation path is represented by the visible length of the skeleton in the surface projection. and distortion ( High twist and multiple branches (high bifurcation point density) mean that the crack consumes a large amount of fracture surface energy as it extends deeper, resulting in a wide surface width but a shallow actual depth.

[0140] Based on this, the present invention constructs a nonlinear fracture proxy mapping model.

[0141] ;

[0142] To support this physical mapping, the following core physical proxy features can be selected or combined:

[0143] (1) Opening proxy quantity: equivalent average width 75th percentile width 90th percentile width Reflects the local maximum opening displacement .

[0144] (2) Extended proxy quantity: overall pixel area and total skeleton length, reflecting the total scale of macroscopic strain energy released by the crack.

[0145] (3) Complexity of proxy quantity: Distortion ( , calculated as the ratio of the actual path to the straight-line distance of the endpoint), branch density, endpoint density, and elongation rate, reflect the complexity of the force pattern and the obstruction of depth-direction expansion.

[0146] Monotonic functions, piecewise functions, saturated functions, or gated functions can be constructed using crack opening proxies, propagation proxies, and complexity proxies to reflect the nonlinear relationship between crack depth and surface opening, propagation, or increased complexity. These functions do not require strict replication of the measured fracture mechanics equations but emphasize the consistency between visual information and mechanistic trends.

[0147] 11. Training Strategies

[0148] For image branches, transfer learning or self-supervised pre-training initialization is used, while for structured feature branches, median interpolation, standardization, and sparse regularization coding are employed. During training, multi-stage learning rate scheduling, progressive resolution training, label smoothing, class or interval reweighting, sample difficulty sampling, and early stopping strategies can be used.

[0149] Set up pseudo-label self-training: first use the main model to generate depth pseudo-labels for unlabeled crack images, and then select high-confidence samples to add to the training; cross-domain adaptation can also be set to align laboratory samples with field samples in the feature space.

[0150] It is also possible to construct a teacher-student structure: using a feature-level fusion head or a multimodal large model as the teacher and an image master head as the student, multi-source knowledge is compressed into a single image deployment model through distillation loss.

[0151] 12. Reasoning Output and Engineering Applications

[0152] During the inference phase, only the main image header is retained, and depth values, depth ranges, and confidence scores are directly output from single images, video frames, or local crack regions. Alternatively, segmentation results can be combined to output crack region highlight maps and local depth annotations.

[0153] The output results are further correlated with risk levels and treatment recommendations. For example, when the predicted depth exceeds a certain threshold range, the system triggers recommendations for re-inspection, grouting, sealing, adhesive reinforcement, steel wrapping, increasing the cross-section, local repair, or key monitoring.

[0154] The results can be written into inspection reports, digital twin platforms, BIM systems, edge terminals, mobile apps, or cloud databases to achieve spatiotemporal tracking and full lifecycle management of crack depth.

[0155] like Figure 6 As shown, based on the above method, it can be practically applied to the following scenarios:

[0156] Example 1: Example of estimating crack depth in laboratory flexural members

[0157] This embodiment constructs a dataset containing crack images, crack masks, and depth labels. Crack images are acquired at different loading stages of the component, and depth labels are obtained through profiling or depth measurement. First, a crack segmentation network is trained to obtain the crack mask, and then structured features such as width statistics, skeleton length, area, and tortuosity are extracted from the mask. Simultaneously, the original images are input into a visual backbone to obtain visual representations. Subsequently, four schemes—decision-level fusion, feature-level fusion, structural constraints, and physical proxy constraints—are run to compare their impact on image master-head stability and generalization.

[0158] In this embodiment, the structural constraint scheme can be expressed as follows: the image master head is supervised by the structural auxiliary head and constrained by the consistency loss during training, while only the image master head is output during inference. The physical surrogate constraint scheme can be expressed as follows: opening surrogate, extended surrogate, and complexity surrogate are selected from the structured features to construct a surrogate comprehensive score, and ranking constraints and feasible region constraints are applied.

[0159] The beneficial effect of this embodiment is that even without specifying the specific network, sample size, and training hyperparameters in the specification, higher stability and interpretability than simple image regression can be obtained through the technical chain of "segmentation enhancement + structural features + visual master head + constraint learning".

[0160] Example 2: Inspection of Existing Bridge or Tunnel Components

[0161] This embodiment acquires images of cracks in the bridge web, bridge deck, tunnel lining, or sidewalls. Weakly supervised labels are constructed using on-site inspection records, historical maintenance records, local non-destructive testing results, and manual experience-based grading. Due to variations in lighting, contamination, texture noise, and viewing angle in the on-site samples, crack segmentation-guided clipping, background suppression, cross-domain adaptation, and confidence rejection modules can be enabled in this invention.

[0162] For cracks where absolute depth is difficult to obtain, a multi-task training framework can be constructed using interval labels, sorting labels, and stage labels. The main image head outputs the depth interval or risk level, while the structural / physical auxiliary head provides constraints during training.

[0163] This embodiment illustrates that the present invention does not rely on a single accurate measurement label, but can still achieve depth estimation for practical engineering scenarios through weak supervision, multi-tasking, and mechanism proxy.

[0164] Example 3: Separation of Single Image Deployment and Multimodal Training Example

[0165] In this embodiment, during the training phase, the original image, crack mask, structured features, and physical surrogate quantity are used for joint training. During the inference phase, only the image master head is retained, thus achieving "training from multiple sources and deploying from a single source." This solution is particularly suitable for scenarios with limited resources, such as mobile terminals, drones, and edge devices.

[0166] Specifically, the fusion head or multimodal teacher network can exist only during training, compressing multi-source knowledge into the image master head through distillation loss or consistency loss. The inference device does not need to compute structured features in real time, nor does it need to be connected to an external physical quantity acquisition device.

[0167] The value of this embodiment lies in highlighting that the present invention is not a simple multimodal system, but an engineered method with the capability of "multi-source training - single-source deployment".

[0168] Example 4: Fracture Mechanics Surrogate Enhancement Example

[0169] In this embodiment, for scenarios with known component dimensions and stress boundary information, crack opening proxy, crack length proxy, and component scale factor can be combined into stress intensity factor proxy or energy release rate proxy, and then trained together with the vision master head. The ranking loss ensures that samples with larger mechanistic proxies correspond to larger depth outputs, while the range loss ensures that the depth does not exceed the geometrically feasible region of the component.

[0170] In further implementation, crack evolution stage labels can be established based on the occurrence sequence, width transition, bifurcation growth, and crack path changes of primary and secondary cracks, so that crack depth prediction and fracture stage identification can be carried out simultaneously.

[0171] This embodiment illustrates that the present invention can embed the "relationship trend" and "stage logic" in fracture mechanics into deep learning in the form of surrogate variables, without requiring precise solution of complex mechanical parameters item by item.

[0172] Example 5: Scalable Network Replacement Example

[0173] In this embodiment, the visual backbone is replaced with a residual network, an efficient network, ConvNeXt, SwinTransformer, or a lightweight mobile network, respectively; the structural encoder is replaced with an MLP, a tree model encoder, a tabular Transformer, an autoencoder, or a graph network encoder; and the fusion unit is replaced with a ridge regression, a gated fusion unit, a cross-attention fusion unit, or a hybrid expert network. As long as the overall idea of ​​"image main task + structured features / physical proxy constraints + depth output" is still satisfied, it belongs to an equivalent form that can be implemented by this invention.

[0174] This embodiment further illustrates that the present invention is not bound by a specific backbone name; the key lies in the module relationship, constraint relationship, and output relationship.

[0175] Example 6: Illustrative Effect Chart Explanation

[0176] Figure 5 A comparative illustration of the combined effects of different technical approaches is provided. This chart does not limit specific experimental values, but rather illustrates the evolutionary relationship and overall performance improvement trend from single image baseline, decision-level fusion, feature-level fusion to structural constraints and physical proxy constraints in this invention. This type of chart can be replaced with measured charts according to different experimental or application scenarios, or it can be retained as a trend illustration.

[0177] In this invention, the effect charts are not limited to error index bar charts, but can also be actual value-predicted value scatter plots, residual distribution plots, absolute error distribution plots, stage comparison plots of different modules, confidence coverage plots, component scene grouping comparison plots, depth interval confusion plots, sorting consistency plots, and risk level heatmaps.

[0178] As can be seen from the above description of the invention and its embodiments, this invention is not simply a crack detection, crack segmentation, crack width measurement, or image regression technology, but rather a unified technical solution that is multi-level, multi-source, interpretable, and deployable, built around the internal damage index of "crack depth." It can be trained in a multimodal manner or deployed in a single-image manner; it can employ statistical structural features or physical proxy constraints; it can directly output depth or output depth intervals, levels, and confidence levels; and it can serve both laboratory specimens and actual engineering inspections.

[0179] Furthermore, the present invention also provides the following embodiments:

[0180] Example 7: Multi-view crack depth estimation example

[0181] In this embodiment, multiple images from different viewpoints, focal lengths, or time points are acquired for the same crack location. First, multi-view associations are established through geometric registration, keypoint matching, or viewpoint encoding. Then, features are extracted using a shared visual backbone or multi-branch backbone, and fused using a viewpoint attention module. This embodiment is suitable for scenes with complex surface textures and significant single-viewpoint occlusion, and can serve as an extension solution for multi-view scenarios.

[0182] Example 8: Video Sequence Crack Depth Estimation Example

[0183] In this embodiment, the input is no longer limited to static images, but can be a video sequence captured while moving along the crack direction. Temporal features are extracted using temporal networks, convolutional recurrent networks, temporal Transformers, 3D convolutional networks, or keyframe fusion networks. This allows the model to utilize redundant information from preceding and following frames to cancel out single-frame noise and estimate depth stability and trend values ​​in time. This embodiment is particularly suitable for path inspection by inspection robots and drones.

[0184] Example 9: Weak Labeling and Semi-Supervised Examples

[0185] In this embodiment, some samples have precise depth labels, while others have only interval labels, rank labels, ranking labels, or no labels. For unlabeled samples, pseudo-labels can be generated using a teacher model and added to the training process after confidence screening. For samples with interval labels, interval consistency loss can be used, and for samples with ranking labels, paired ranking loss can be used. This embodiment can significantly reduce annotation costs and provides a textual basis for the subsequent extension of the patent to practical engineering scenarios.

[0186] Example 10: Material and Scene Transfer Example

[0187] In this embodiment, the source domain can be a flexural crack in reinforced concrete, and the target domain can be a masonry crack, pavement crack, metal surface crack, interlayer crack in composite material, or rock fissure. The model is migrated from the source domain to the target domain using a domain adaptation module, style transfer module, adversarial alignment module, or statistical recalibration module. Since this invention protects a "generalized technical framework for crack depth estimation," this embodiment is advantageous for expanding the scope of applications.

[0188] Example 11: Active Learning Example

[0189] In this embodiment, the system first provides depth predictions and uncertainty scores for unlabeled images. Then, it prioritizes pushing high-uncertainty, boundary samples, and samples with prediction conflicts to labelers for review, thereby forming high-value incremental data. This allows for continuous iterative improvement of the model and is naturally compatible with closed-loop management of engineering inspections.

[0190] Example 12: Maintenance Decision-Making Linkage Example

[0191] In this embodiment, the predicted depth is not only displayed as a single numerical value, but also further incorporated into the maintenance strategy decision-making module, which outputs suggestions such as "continue monitoring," "surface sealing," "grouting repair," "adhere fiber materials," "external steel reinforcement," "increase the cross-section," and "partial removal and recasting." Even if this module does not constitute a final independent claim, it can serve as supporting content for the technical effect and application completeness in the patent specification.

[0192] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any modifications or equivalent changes made based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A method for estimating crack depth based on crack visual images, structured feature constraints, and physical constraints, characterized in that, Includes the following steps: S1. Acquire crack images, video frames, or multimodal images of the target component; S2. Perform one or more preprocessing steps on the crack image, video frame or multimodal image, including crack detection, crack segmentation, crack region enhancement, boundary reconstruction, skeleton extraction or noise suppression. S3. Extract the structured features of the cracks from the pre-processed crack region; S4. Input crack images, video frames, or multimodal images into a visual feature extraction network to form a visual depth representation; S5. Based on the crack structured features, visual depth representation, and at least one fusion or constraint module, output the crack depth value, crack depth range, crack depth level, or depth-related damage index.

2. The crack depth estimation method based on crack visual images, structured feature constraints, and physical constraints according to claim 1, characterized in that, The extracted crack structured features in step S3 include any one or any combination of the following: average width, median width, minimum width, maximum width, equivalent average width, quantile width, width dispersion, width coefficient of variation, width skewness, width kurtosis, crack area, crack perimeter, skeleton length, number of endpoints, number of bifurcation points, endpoint spacing, tortuosity, branch density, endpoint density, bounding box width, bounding box height, aspect ratio, convex hull area, solidity, compactness, area-to-perimeter ratio, principal orientation angle, major axis dispersion, minor axis dispersion, elongation, and number of connected components.

3. The crack depth estimation method based on crack visual images, structured feature constraints, and physical constraints according to claim 1, characterized in that, The visual feature extraction network in step S4 is any one or any combination of several of the following: convolutional neural network, residual network, dense connection network, MobileNet-like network, EfficientNet-like network, RepVGG-like network, ConvNeXt-like network, Vision Transformer, Swin Transformer, hybrid convolutional-transformer network, state space visual network, image pyramid network, and temporal visual network.

4. The crack depth estimation method based on crack visual images, structured feature constraints, and physical constraints according to claim 1, characterized in that, The fusion or constraint module in step S5 can be any one of the following: decision-level fusion module, feature-level fusion module, structured feature constraint module, and physical proxy constraint module.

5. The crack depth estimation method based on crack visual images, structured feature constraints, and physical constraints according to claim 1, characterized in that, The method also includes determining the stage of crack evolution, specifically: Based on the relationship between the main crack and the secondary crack, the crack end extension state, the width-length coupling state, the bifurcation morphology, the abrupt change in direction, or the load history label, the crack stage category is output, and the crack stage category is introduced into the depth estimation process as additional supervision information, prior constraint information, or gating information.

6. The crack depth estimation method based on crack visual images, structured feature constraints, and physical constraints according to claim 1, characterized in that, The method also includes uncertainty assessment, specifically: Prediction confidence, confidence intervals, or rejection flags can be generated through deep integration, Dropout, quantile regression, Bayesian regression head, error prediction head, confidence calibration layer, or distributed regression.

7. The crack depth estimation method based on crack visual images, structured feature constraints, and physical constraints according to claim 1, characterized in that, The method also includes: Based on fracture mechanics surrogate modeling, a mapping relationship is established between any one or more of the following: stress intensity factor surrogate, double K fracture criterion surrogate, energy release rate surrogate, crack surface opening displacement surrogate, equivalent crack bandwidth surrogate, structural size correction factor, load location factor, support boundary factor, or material brittleness factor, and the mapping relationship is established with crack structure characteristics. This mapping relationship is then used to constrain or correct the final depth prediction.

8. A crack depth estimation system based on crack visual images, structured feature constraints, and physical constraints, characterized in that, include: The data acquisition module is used to acquire crack images or video frames of the target component; The crack segmentation and enhancement module is used for preprocessing crack images; The structured feature extraction module is used to extract the structured features of cracks; The visual depth representation module is used to extract visual depth representations from crack images; A multi-level fusion or constraint module is used to fuse the structured features and visual depth representation and apply constraints; The depth output module is used to output crack depth values, ranges, grades, or damage indicators. Each module is configured to perform the method described in any one of claims 1 to 7.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program stored in the memory and executable on the processor; when the program is executed by the processor, it implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, implements the method described in any one of claims 1 to 7.

Images