Geometric prior embedded building multi-type defect image rapid detection method and system

Abstract

The present application provides a geometric prior embedding building multi-type defect image rapid detection method and system, belonging to the technical field of building defect detection. The present application is to solve the problem that the building defect detection is easily affected by monocular depth scale uncertainty, image enhancement misplacement and feature distribution mutation, leading to unstable training or decreased detection performance. The method comprises: constructing a defect detection sample. Constructing a multi-scale building apparent defect detection backbone network; constructing a pixel-level geometric prior extraction branch; aligning and fusing the geometric prior features and multi-scale detection features. Channel re-labeling is performed on the fused multi-scale detection features. An end-to-end training and reasoning method is established. According to the engineering application scene, the detection result is generated and the system deployment is completed. The system includes an image input module, a main detection module, a geometric prior module, a geometric-semantic fusion module, a channel re-labeling module, a training optimization module, a reasoning deployment module and a result output module.

Description

Technical Field

[0001] This invention relates to a method and system for rapid detection of multiple types of building defects using geometric prior embedding, belonging to the field of building defect detection technology. Background Technology

[0002] Buildings often exhibit visible defects such as concrete spalling, wall damage, exposed rebar, and rebar buckling after earthquakes, fires, freeze-thaw cycles, long-term service, material aging, and construction disturbances. These defects not only affect the building's appearance and durability but may also reflect safety hazards such as protective layer failure, reduced local load-bearing capacity, increased risk of rebar corrosion, and degradation of component ductility. Therefore, timely and accurate identification of building surface defects is a crucial foundation for structural safety assessments, post-disaster emergency inspections, and repair and reinforcement decisions.

[0003] Traditional building defect inspections mainly rely on manual visual inspection, photographic recording, and experience-based judgment, resulting in low inspection efficiency. The image-based automatic defect detection method faces challenges such as strong subjectivity, discontinuous recording, difficulty in accessing dangerous areas, and difficulty in quickly verifying large-scale samples. With the application of drones, mobile terminals, and edge computing devices in civil engineering inspections, image-based automatic defect detection methods have gradually become an important technical approach. However, actual inspection images often have characteristics such as unstable shooting angles, large changes in lighting, shadow occlusion, complex background textures, motion blur, and significant differences in target scale, posing significant challenges to general target detection algorithms.

[0004] Existing single-stage detectors based on convolutional neural networks offer high speed, but are susceptible to local texture interference in complex structural contexts and typically rely on post-processing steps such as nonmaximum suppression. The DETR series of methods models object detection as an end-to-end ensemble prediction problem, possessing global relation modeling capabilities; RT-DETR, DEIM, and DEIMv2 further improve real-time detection efficiency and training convergence speed. Among them, DEIMv2 utilizes DINOv3-style visual representations and a spatial tuning adapter to transform single-scale Transformer features into multi-scale detection features, providing a better real-time detection foundation for building defect detection.

[0005] However, building defects are not simply two-dimensional texture targets. Concrete spalling is usually accompanied by local depressions, fracture boundaries, and changes in surface normals; wall damage often has irregular areas and material fracture edges; exposed rebar presents as a slender, connected structure; and rebar buckling manifests as bending of the exposed rebar's centerline, local curvature anomalies, and spatial offset. When relying solely on RGB appearance, shadow boundaries, stains, construction textures, and real defects are easily confused. Existing methods that directly stitch together depth maps or use depth maps as additional input are susceptible to the effects of monocular depth scale uncertainty, image enhancement misalignment, and abrupt changes in feature distribution, leading to unstable training or decreased detection performance. Summary of the Invention

[0006] This invention addresses the problem that building defect detection is easily affected by monocular depth scale uncertainty, image enhancement misalignment, and abrupt changes in feature distribution, leading to unstable training or decreased detection performance. Therefore, it proposes a fast detection method and system for multi-type building defect images using geometric prior embedding.

[0007] The technical solution adopted by this invention to solve the above problems is: the fast detection method for multiple types of building defects using geometric prior embedding proposed in this invention, comprising: Step 1: Collect images of building defects after the earthquake, define the categories of the defect images and label them with rectangular boxes to construct defect detection samples; Step 2: Construct a multi-scale building appearance defect detection backbone network. Input the defect samples into the multi-scale building appearance defect detection backbone network and output multi-scale detection semantic features, which include defect categories and bounding box prediction results. Step 3: Input the defect samples from the multi-scale building appearance defect detection backbone network into the geometric prior extraction branch to obtain geometric prior features; Step 4: Align and fuse the geometric prior features with the multi-scale detection semantic features to obtain the geometric-semantic fusion multi-scale detection semantic features; Step 5: Perform channel recalibration on the multi-scale detection semantic features after geometry-semantic fusion to enhance the channel responses related to the defined defect categories; Step 6: Based on steps 2-5, implement end-to-end training and inference. During the training phase, perform end-to-end optimization of trainable parameters. During the inference phase, disable data augmentation and dense supervision strategies on the training side and output the category, confidence, and bounding box for each building defect instance. Step 7: Based on the actual application requirements, select the corresponding estimation method to generate building inspection results, visualization annotation map and defect statistics, and output the inspection results to the system terminal.

[0008] Furthermore, step 1 specifically includes: Step 1.1: Obtain defect images of post-earthquake buildings, concrete components, facades, and interior and exterior structural surfaces using one of the following methods: drone inspection, mobile terminal photography, fixed camera equipment acquisition, or engineering inspection database. The defect images are RGB images. Step 1.2: Identify the defect category in the defect image, where the defect categories include concrete spalling, wall damage, exposed rebar, and rebar buckling; concrete spalling indicates irregular areas formed by the detachment of the protective layer / surface material; wall damage indicates cracks, gaps, and material deterioration on the surface of walls, masonry, or components; exposed rebar indicates slender targets formed by the exposed rebar after the protective layer is damaged; rebar buckling indicates a geometric state in which the centerline of exposed rebar is bent, locally increased in curvature, or laterally offset; Step 1.3: Establish the correspondence between image number, file name, width and height, category number, bounding box coordinates, area and instance number according to the general object detection data format, and uniformly represent the annotation box in the form of [x,y,w,h].

[0009] Furthermore, the multi-scale building appearance defect detection backbone network in step 2 includes a lightweight visual feature extraction sub-network, a detection scale conversion sub-network, a hybrid coding module, and a decoder connected in sequence. The lightweight visual feature extraction subnetwork is obtained through DINOv3 visual representation distillation and is used to encode images in the input defect detection samples and extract single-scale semantic features. The detection scale transformation subnetwork adopts a spatially tuned adaptation structure, which includes a main detection module, parameter-free bilinear resampling, a lightweight convolutional detail branch, and a multi-scale fusion structure to simultaneously preserve high-resolution details and deep semantic information. The main detection module includes an upsampling branch, an identity mapping branch, and a downsampling branch. The detection scale transformation subnetwork is used to convert single-scale semantic features into three detection scales: P3, P4, and P5. The hybrid encoding module is used for bidirectional cross-scale feature fusion and selects high-quality candidates as the initial target query. The decoder is used to iteratively refine the initial target query and output multi-scale detection semantic features.

[0010] Furthermore, step 2 specifically includes: Step 2.1: Encode the images in the defect detection samples based on a lightweight visual feature extraction network, and extract the multi-scale semantic features of the corresponding images. ; Step 2.2: Multi-scale semantic features Identity mapping is performed to obtain P4 detection scale features. Lightweight convolutional detail branches are used to set up the convolutional structures for upsampling and downsampling branches. The upsampling branch employs two 3D layers. 3 convolutions, with the downsampling branch using two layers of 3 3 convolutions are used to obtain features at three detection scales, P3, P4, and P5, through upsampling, identity mapping, and downsampling branches. Step 2.3: Align the features of the three detection scales P3, P4, and P5 through 1×1 input projection to complete the channel alignment; Step 2.4: Input the channel-aligned features into the hybrid encoding module. The low-resolution deep semantic information is transferred to the high-resolution features through the corresponding scale convolution fusion function and the top-down path. The details and localization information in the high-resolution features are fed back to the low-resolution features through the corresponding scale convolution fusion function and the bottom-up path, thus completing the bidirectional cross-scale fusion and obtaining features E3, E4 and E5 containing deep semantic information as well as defect boundaries, locations and scales. Step 2.5: Select features E3, E4 and E5 as candidates, and select candidates with quality higher than the preset value as the initial target query and transmit them to the decoder. Use multi-scale encoded features as keys and values ​​to iteratively refine the initial target query. The classification prediction head and bounding box regression prediction head output multi-scale detection semantic features. Multi-scale semantic features The calculation formula is: (1); In formula (1), The input image has dimensions H×W×3. Here is a visual feature extraction network with parameter θb; cb represents the number of feature channels; H and W represent the height and width of the input defect detection image, respectively.

[0011] Furthermore, step 3 specifically includes: A pre-trained monocular geometric estimation model is used as the geometric prior extraction branch; wherein, the geometric prior extraction branch keeps the parameters frozen during training and only performs forward inference; the gradient from the detection loss is not backpropagated to the geometric prior extraction branch; The input image is mapped to a pixel-aligned depth map D and a ray map M, where the ray map records the ray origin t and ray direction d corresponding to each pixel; Based on the pixel coordinates, ray origin coordinates, and unit ray direction vector, the three-dimensional spatial point P corresponding to the pixel is recovered by combining the depth map D and the ray map M. Robust normalization is performed on the depth map, and the horizontal and vertical depth gradients are calculated using the difference operator; The surface normal is estimated based on the spatial variation of adjacent three-dimensional points, and the local curvature is calculated based on the variation of the surface normal. The geometric prior features G are obtained by combining the robustly normalized depth ray direction, depth gradient, surface normal, and local curvature.

[0012] Furthermore, step 4 specifically includes: For each detection scale of multi-scale detection semantic features, the geometric prior feature G is size-aligned according to the current semantic feature resolution and mapped to the same channel dimension as the semantic feature through 1×1 convolution / linear projection. Using the corresponding scale's detection semantic features as the query and the linearly projected geometric prior features as the key and value, cross-attention is calculated. ; Set learnable residual coefficients By controlling the injection intensity of geometric prior features at each detection scale, cross-attention is added to the original semantic features in a residual manner to complete alignment and fusion, thereby obtaining multi-scale detection semantic features after geometric-semantic fusion. Among them, learnable residual coefficients Initially set to 0 or a small value close to 0, the model approximates the original detector in the early stages of training; as training progresses, the network adaptively adjusts based on the reliability of geometric information. ; Multi-scale detection of semantic features The calculation formula is: (2); In formula (2), The first before the integration i Scale semantic features.

[0013] Furthermore, step 5 specifically includes: Multi-scale detection semantic features after geometric-semantic fusion The input hybrid coding module is subjected to cross-scale interaction and fusion. After the fusion output at each scale, a channel recalibration structure is set to select channels for the detection features that have aggregated semantic and localization information. Global average pooling is performed on the fused multi-scale detection semantic features to obtain a channel-level global description. Channel weights are generated through two layers of lightweight mapping and nonlinear activation. Based on channel weight Multi-scale detection semantic features after geometry-semantic fusion Channel-by-channel multiplication recalibration is performed, where the compression ratio of the channel recalibration structure is set according to the number of detection feature channels. Let C be the number of detection feature channels of the fused multi-scale detection semantic features, and r be the compression ratio of the channel recalibration structure, satisfying the following conditions: ,in, The preset baseline number of channels; the two-layer lightweight mapping includes a first lightweight mapping layer and a second lightweight mapping layer. The first lightweight mapping layer reduces the number of channels from... Compress to The second lightweight mapping layer increases the number of channels from... Restore to To generate channel weights corresponding to each channel of the detected features; By recalibrating the channels, we can assess the channel responses related to concrete spalling edges, damaged wall areas, exposed slender rebar structures, and local curvature of rebar buckling, while suppressing redundant channels corresponding to wall textures, shadow boundaries, stains, and edges of non-defective components.

[0014] Furthermore, step 6 specifically includes: During the training phase, steps 2-5, the spatial tuning and adaptation structure, the geometric projection layer, the cross-attention layer, the residual scaling factor, the channel recalibration structure, the detection decoding structure, the classification prediction head, and the bounding box regression prediction head are used as the trainable parts; the pre-trained geometric prior extraction branch is used as the frozen part. End-to-end optimization is performed using dense one-to-one matching supervision, matching quality-aware classification loss, bounding box regression loss, and auxiliary layer supervision. During the inference phase, the training-side data augmentation and dense supervision strategies are turned off. Based on the input building defect inspection images, multi-scale semantic detection feature extraction, pixel-level depth-ray geometric prior feature extraction, geometric-semantic fusion, channel recalibration, and detection decoding are performed sequentially to output the category, confidence level, and bounding box of each building defect instance.

[0015] Furthermore, step 7 specifically includes: Based on the actual application requirements, select one of the following methods to generate building defect detection results, visualization annotation maps and defect statistics: geometric feature caching, keyframe geometric estimation, online geometric estimation or lightweight geometric estimation, and output the detection results to the system terminal.

[0016] Furthermore, this invention also proposes a fast image detection system for multiple types of building defects using geometric prior embedding, comprising: Image input module, used to receive building inspection images; The main detection module is used to generate the multi-scale semantic detection features based on the building inspection images; The geometric prior module is used to generate the pixel-level depth-ray geometric prior features based on the building inspection images; The geometry-semantic fusion module is used to perform the scale alignment and geometry-semantic fusion based on the multi-scale semantic detection features and the pixel-level depth-ray geometric prior features; The channel recalibration module is used to perform the channel recalibration on the fused multi-scale detection features. The training optimization module is used to perform the end-to-end training; The inference deployment module is used to schedule geometric feature caching, keyframe geometric estimation, or online geometric estimation to perform inference based on real-time inspection, edge deployment, or offline analysis requirements. The results output module is used to output the category, confidence level, bounding box coordinates, and visualization results of building defects.

[0017] The beneficial effects of this invention are: 1. By introducing the depth-ray geometric prior output by the pre-trained monocular geometric estimation model, this invention can supplement the spatial structural information in addition to the texture of the three-channel red, green and blue images for building defect detection, making it easier for the model to focus on concrete spalling boundaries, wall damage areas, exposed slender structures of steel bars, and local curvature anomalies of steel bar buckling.

[0018] 2. This invention, through a residual geometric-semantic fusion structure, avoids abrupt changes in feature distribution and enhancement misalignment interference caused by directly stitching together depth maps. It enables geometric information to participate in multi-scale detection feature representation in a learnable, suppressable, and enhanceable manner, thereby improving the localization stability in complex scenes.

[0019] 3. By setting a channel recalibration structure on the multi-scale detection features after cross-scale interaction and fusion in the hybrid coding structure, this invention can enhance the response of defect-related channels, suppress false activations caused by wall textures, shadows, stains and non-target edges, thereby reducing false detections and improving the detection capability of small and slender targets. Attached Figure Description

[0020] Figure 1 A flowchart of a fast detection method for multiple types of building defects using geometric prior embedding; Figure 2 A schematic diagram showing the samples and categories of building defects; Figure 3 This is a schematic diagram illustrating the category distribution of the building defect dataset. Figure 4 A schematic diagram of the main inspection process for building surface defects and the spatial tuning and adaptation structure. Figure 5 A schematic diagram of the structure for extracting branches and constructing compact geometric features based on geometric priors; Figure 6 This is a schematic diagram of the geometry-semantics fusion structure; Figure 7 Schematic diagram of channel recalibration structure and insertion position; Figure 8 This is a schematic diagram of the training and inference modes; Figure 9 This is a general framework diagram of a building defect detection system. Figure 10This is a comparison chart of the training AP convergence curves of this invention and other models; Figure 11 This is a comparison of the AP-small convergence curves of the present invention with those of other models. Detailed Implementation

[0021] Specific implementation method one: as follows Figure 1 As shown, the steps of the fast detection method for multiple types of building defects using geometric prior embedding described in this embodiment include: S1: Acquire images of building defects and construct defect detection samples.

[0022] S101: Collect images of defects in buildings, concrete components, facades, and interior and exterior structural surfaces after the earthquake. Image sources may include drone inspections, mobile terminal photography, fixed camera equipment, or existing engineering inspection databases. A sample diagram of building defects and their categories is shown below. Figure 2 As shown; S102: Defects are categorized into four types: concrete spalling, wall damage, exposed rebar, and rebar buckling. Concrete spalling refers to irregular areas formed by the detachment of the protective layer or surface material; wall damage refers to cracks, gaps, and material deterioration on the surface of walls, masonry, or structural members; exposed rebar refers to slender targets formed by the exposed rebar after the protective layer is damaged; and rebar buckling refers to the geometric state of exposed rebar where the centerline is bent, the local curvature is increased, or there is a lateral shift.

[0023] S103: Establish the correspondence between image number, file name, width and height, category number, bounding box coordinates, area and instance number according to the general object detection data format, and uniformly represent the annotation box in the form of [x,y,w,h] so that the cross-union ratio can be calculated during the training phase, and the average precision, average precision under different cross-union ratio thresholds, average precision of small targets and average recall when there are a maximum of 100 detection results can be calculated during the evaluation phase.

[0024] S104: To address issues such as imbalanced sample classes, scarcity of small targets, and interference from complex lighting and textures, enhancement strategies such as random flipping, color perturbation, image stitching, copy-paste, or weak blending are introduced during the training phase. Defect category, size, and geometric attribute statistics are preserved for subsequent model training and error analysis. The distribution of building defect dataset categories is illustrated below. Figure 3 As shown.

[0025] S2: Construct a multi-scale building appearance defect detection backbone network; The main inspection process for building appearance defects is as follows: Figure 4 As shown, it includes: S201: The input three-channel red-green-blue (RGB) image is fed into a lightweight visual feature extraction network obtained by DINOv3 visual representation distillation to extract single-scale visual transformer features with global semantic expression capabilities.

[0026] The visual feature extraction network encodes the input image, and the extraction process of single-scale semantic features is represented as follows: (1); In formula (1), The input image has dimensions H×W×3. Here is a visual feature extraction network with parameter θb; cb represents the number of feature channels; H and W represent the height and width of the input defect detection image, respectively.

[0027] S202: The single-scale visual transformer features are converted into three detection scales (P3, P4, and P5) through a spatially tuned adaptation structure. For example... Figure 4 As shown, the spatial tuning adaptation structure includes a main detection module, parameter-free bilinear resampling, lightweight convolutional detail branches, and a multi-scale fusion structure, which are used to simultaneously preserve high-resolution details and deep semantic information.

[0028] The main detection module constructs three detection scales through an upsampling branch, an identity mapping branch, and a downsampling branch. Its calculation process is as follows: (2); (3); (4); (5); In formulas (2)-(5), , and These represent detection features with step sizes of 8, 16, and 32, respectively. This indicates double bilinear upsampling; and This represents two 3×3 convolutions in the upsampling branch; and This represents two 3×3 convolutions in the downsampling branch; s represents the convolution stride. This represents the GELU activation function; It is obtained from Fb through identity mapping.

[0029] S203: Channel alignment is achieved by projecting the three scale features through a 1×1 input, and the channel alignment process is expressed as follows: (6); In formula (6), represents the feature after channel alignment at the i-th scale; wi and bi represent the weights and biases of the 1×1 convolution at the i-th scale, respectively; the symbol * represents the convolution operation; cs represents the number of feature channels after unification.

[0030] The channel-aligned features are fed into the hybrid coding structure. First, low-resolution deep semantic information is passed to high-resolution features through a top-down path. The calculation process is as follows: (7); In formula (7), This represents the feature at the i-th scale after top-down fusion; Let i represent the convolution fusion function at the i-th scale; (·) indicates double upsampling.

[0031] Subsequently, details and localization information from the high-resolution features are fed back to the low-resolution features via a bottom-up path. The calculation process is as follows: (8); In formula (8), This represents the encoded feature of the i-th scale after bidirectional cross-scale fusion; This represents the convolutional fusion function at the corresponding scale; Down2(·) represents double downsampling. The resulting E3, E4, and E5 simultaneously contain deep semantic information as well as detailed information such as defect boundaries, locations, and scales.

[0032] S204: The encoding structure generates candidate classification and localization results, and selects the higher-quality candidates as the initial target query; the simplified deformable transformer decoding structure uses multi-scale encoded features as keys and values ​​to iteratively refine the target query, and finally outputs the detection results by the classification prediction head and the bounding box regression prediction head.

[0033] S3: Construct pixel-level geometric prior extraction branches; Geometric prior extraction branches and compact geometric feature construction structure, such as Figure 5 As shown, its workflow includes: S301: A pre-trained monocular geometry estimation model (DepthAnything3, or DA3 for short) is used as the geometry prior extraction branch to map the input image into a pixel-aligned depth map D and ray map M, where the ray map records the ray origin t and ray direction d corresponding to each pixel.

[0034] S302: In depth-ray representation, the three corresponding pixels are recovered by comparing the depth map and the ray map. A point P in a 3D space is calculated as follows: (9); In formula (9), Represents pixel coordinates, This represents the three-dimensional point corresponding to the pixel. Indicates the origin of the ray. Represents the normalized depth value. The symbol represents the direction vector of the unit ray, and the dot sign indicates that a scalar is multiplied by a vector.

[0035] S303: Performs robust normalization on the depth map, calculates the horizontal and vertical depth gradients using the differential operator; estimates the surface normal based on the spatial variation of adjacent 3D points, and calculates the local curvature or deformation intensity based on the normal variation or the local point cloud neighborhood variation.

[0036] S304: Combine normalized depth, ray direction, depth gradient, surface normal, and local curvature into a compact geometric feature G. The geometric prior extraction branch keeps its parameters frozen during training and only performs forward inference; gradients from the detection loss are not backpropagated to this branch, thereby preserving the general spatial prior of the large-scale pre-trained geometric model and reducing the risk of unstable fine-tuning caused by small sample defective data.

[0037] S4: Align and fuse geometric prior features with multi-scale detection features.

[0038] S401: As Figure 6 As shown, for each detection scale, the compact geometric features G output by the geometric prior extraction branch are sized according to the current semantic feature resolution and mapped to the same channel dimension as the semantic features through 1×1 convolution or linear projection.

[0039] S402: Using detected semantic features as the query and projected geometric features as the key and value, cross-attention is calculated to enable the detected semantic feature tokens to adaptively retrieve geometric evidence such as depth gradient, ray direction, surface normal, and local curvature. The cross-attention calculation method is as follows: (10); In formula (10), i represents the detection scale number, Qi represents the query matrix obtained from the semantic features of the i-th scale, ki and vi represent the key matrix and value matrix obtained from the geometric features of the scale, respectively, d represents the key vector dimension, Softmax represents the normalized exponential function, and Attention represents the cross-attention output.

[0040] S403: Set Learnable Residual Factors The injection intensity of geometric information at each scale is controlled, and the cross-attention results are added to the original semantic features as residuals. The calculation method is as follows: (11); In formula (11), This represents the semantic features at the i-th scale before fusion. This represents the i-th scale feature after fusion. This represents the learnable residual coefficients at that scale. This represents the geometric augmentation amount calculated from the semantic query and the geometric key value.

[0041] S404: Set the residual coefficient The initial value is set to 0 or a small value close to 0, so that the model approximately degenerates into the original detector in the early stage of training. As training progresses, the network adaptively adjusts λi according to the reliability of geometric information, thereby enhancing defect boundaries and local deformations when geometric information is effective, and reducing interference when geometric information is affected by noise or enhancement misalignment.

[0042] S5: Channel recalibration is performed on the fused multi-scale detection features.

[0043] Channel recalibration structure and insertion position as follows Figure 7 As shown, it includes: S501: The multi-scale detection features after geometric-semantic fusion are fed into the hybrid coding structure to complete cross-scale interaction and fusion. After the fusion output of each scale, a channel recalibration structure is set to select channels for the detection features that have aggregated semantic and localization information.

[0044] S502: Perform global average pooling on the input features to obtain a channel-level global description, followed by two layers of lightweight mapping and non-mapping. Linear activation generates channel weights, and then the original features are recalibrated through channel-by-channel multiplication. The calculation method is as follows: (12); (13); In formulas (12) and (13), This represents the channel description obtained by global average pooling of the features at scale i. and This represents a two-layer learnable mapping matrix. Represents a non-linear activation function. Represents an S-shaped gated function. Indicates channel weight, Indicates the features before recalibration. Indicates the features after recalibration. This indicates channel-by-channel multiplication.

[0045] S503: Enhances channel response related to concrete spalling edges, wall damage areas, exposed slender rebar structures, and local curvature of rebar buckling by channel recalibration structure, while suppressing redundant channels corresponding to wall textures, shadow boundaries, stains, and edges of non-defective components.

[0046] S504: When the number of detection feature channels is 192, the compression ratio of the channel recalibration structure is preferably 4 to avoid excessive compression leading to a decrease in the expressive power of defect-related channels; the compression ratio can also be adjusted according to the model size and the computing power of edge devices. Let the number of detection feature channels of the fused multi-scale detection semantic features be C, and the compression ratio of the channel recalibration structure be r, and satisfy... ,in, The preset baseline number of channels; the two-layer lightweight mapping includes a first lightweight mapping layer and a second lightweight mapping layer. The first lightweight mapping layer reduces the number of channels from... Compress to The second lightweight mapping layer increases the number of channels from... Restore to This generates channel weights corresponding to each channel of the detected features.

[0047] S6: Establish end-to-end training and inference methods.

[0048] Training and reasoning models and their differences, such as Figure 8 As shown, it includes: S601: The main detection process for building appearance defects, spatial tuning and adaptation structure, geometric projection layer, cross-attention layer, residual scaling factor, channel recalibration structure, detection decoding structure, classification prediction head and bounding box regression prediction head are used as trainable parts; the pre-trained geometric prior extraction branch is used as the frozen part.

[0049] S602: DenseO2O training logic is used to increase the number of targets in a single image and the density of one-to-one matching supervision; the classification part uses matching quality-aware loss to correlate prediction confidence with matching quality; the localization part uses L1 loss, generalized intersection-over-union (GIoU) ​​loss and auxiliary layer supervision for joint optimization.

[0050] S603: Each training round inputs a batch of building defect images and annotations. First, the main detection process extracts multi-scale semantic features, and at the same time, the frozen geometric prior extracts branches to generate depth-ray geometric features. Then, geometric-semantic fusion and channel recalibration are used to obtain enhanced detection features. Finally, classification, localization and auxiliary supervision losses are calculated, and trainable parameters are updated through backpropagation.

[0051] S604: During the inference phase, training-side data augmentation and intensive one-to-one augmentation strategies are disabled, retaining only the main detection process, geometric prior extraction, geometric-semantic fusion, channel recalibration, decoding, and prediction. The system outputs the category, confidence score, bounding box coordinates, and visualization results for each defect instance.

[0052] S605: In real-time inspection or edge device deployment, geometric features of images or keyframes to be detected can be pre-cached; in accuracy-priority offline analysis, the complete geometric prior extraction branch can be run online; in computing-constrained scenarios, a lightweight geometric coding structure or knowledge distillation branch can be used to replace the complete geometric estimation model.

[0053] S7: Generate detection results based on engineering application scenarios and complete system deployment.

[0054] Depending on the application requirements such as real-time inspection, edge deployment, or offline fine analysis, the system selects geometric feature caching, keyframe geometric estimation, online geometric estimation, or lightweight geometric estimation to generate building defect detection results, visual annotation maps, and defect statistics. The detection results are then output to the inspection report system, post-disaster assessment system, or structural health monitoring platform.

[0055] Specific implementation method two: such as Figure 9 As shown in this embodiment, the geometric prior embedding-based rapid image detection system for multiple types of building defects includes an image input module, a main detection module, a geometric prior module, a geometric-semantic fusion module, a channel recalibration module, a training optimization module, an inference deployment module, and a result output module.

[0056] The image input module receives building inspection images, image metadata, and optional camera parameters; the main detection module generates multi-scale semantic detection features; the geometric prior module generates depth maps, ray maps, and derived geometric features; the geometric-semantic fusion module performs cross-modal mid-layer fusion; the channel recalibration module performs channel selection for the fused detection features; the training optimization module performs end-to-end training; the inference deployment module schedules geometric branches according to online, semi-online, or offline modes; and the result output module generates detection boxes, category labels, confidence scores, defect count statistics, and visualization reports.

[0057] The system can be deployed on workstations, servers, drone inspection platforms, edge computing boxes, or mobile terminals; it can also be connected to building structural health monitoring platforms, post-disaster emergency investigation platforms, or engineering inspection management systems to realize the storage, verification, statistics, and maintenance decision support of building defect detection results.

[0058] Example First, images of building surfaces were acquired, and a detection dataset was established according to four types of defects: concrete spalling, wall damage, exposed rebar, and rebar buckling. In this example, the training set contained 1057 images and 3568 instances, while the validation set contained 253 images and 884 instances, averaging approximately 3.40 targets per image. Each image was formatted according to a common object detection data format, including image information, annotations, and categories. The annotations field recorded the category, bounding box (bbox), area, and instance number, used for subsequent training and evaluation.

[0059] In the model construction phase, DEIMv2-S was selected as the main detector. The input image resolution was preferably set to 640×640. The three-channel red, green, and blue images were first processed by a DINOv3-style visual feature extraction network to extract high semantic features at a resolution of 1 / 16. Then, a spatially tuned adaptation structure was used to form three detection scales with strides of 8, 16, and 32. The features at each scale were channel-aligned and then input into a hybrid encoding structure. The encoding structure output candidate classification and localization results, and selected the 300 candidates with the highest quality as the initial target query. The decoding structure performed multiple rounds of iterative refinement on the query, outputting the final detection box and class confidence score.

[0060] Meanwhile, the frozen pre-trained monocular geometry estimation branch performs forward inference on the same input image, outputting pixel-aligned depth and ray maps. The 3D points corresponding to the pixels are recovered according to formula (9), and the depth gradient, surface normal, and local curvature are calculated. These geometric cues are combined with normalized depth and ray directions to form compact geometric features. Since the parameters of the geometric prior extraction branch remain frozen, the model training process only updates the main detection process, the geometry-semantic fusion structure, the channel recalibration structure, and the prediction head, avoiding the unstable impact of small-scale defect data on the geometry estimation model.

[0061] In the geometric-semantic fusion stage, compact geometric features are aligned to the three detection scales P3, P4, and P5, respectively, and projected to the same channel dimension through 1×1 convolution. Semantic detection features at the corresponding scale are used as queries, and geometric features are used as keys and values ​​for cross-attention calculation. Then, semantic features are injected using learnable residual coefficients according to formula (11). This approach allows detection branches to utilize spalling edges, rebar direction, normal abrupt changes, and local curvature when needed, reducing the impact of geometric branches when geometric quality is poor.

[0062] In the channel recalibration stage, the output of the geometric-semantic fusion structure is fed into a hybrid encoder to complete cross-scale interaction and fusion. A channel recalibration structure is inserted after the fusion output at each scale. This structure first obtains channel descriptions through global average pooling, then obtains channel weights through two layers of mapping and a sigmoid gating, and finally performs channel-by-channel recalibration on the fused features. In this embodiment, the number of hidden channels is 192, and the compression ratio is preferably 4 to balance parameter overhead and channel filtering capability.

[0063] During the training phase, the AdamW optimizer was used with an initial learning rate of 1e-4, weight decay of 1e-4, a batch size of 16, and 132 training epochs. Training augmentations included random flipping, color perturbation, image stitching (Mosaic), copy-paste, and weak mixing. Considering the potential local misalignment between the geometric image and the strongly augmented input image, the residual coefficient of the geometry-semantic fusion structure was set to a small value in the early stages of training, gradually releasing the geometric contribution in later stages. The loss function included matching quality-aware classification loss, L1 bounding box loss, generalized intersection-union (OCU) loss, and auxiliary layer supervision.

[0064] During the inference phase, the system reads images of the buildings to be inspected, generates multi-scale semantic features for the main detection process and geometric features for the geometric prior extraction branch, and outputs defect detection boxes after geometric-semantic fusion and channel recalibration. For offline inspections, the complete geometric prior extraction branch can be run online to obtain more comprehensive geometric priors; for batch inspections by drones or real-time scenarios at the edge, geometric features can be pre-cached or geometric information can be calculated only for keyframes to reduce online inference overhead. The output results include defect category, location, confidence level, defect quantity statistics, and a visual annotation map, which can be further integrated into an engineering inspection report system.

[0065] In addition, to verify the accuracy of building defect detection of the present invention, the following simulation verification was performed in this embodiment: In the verification of the embodiments, the comparison results are shown in Table 1. The average precision (AP) of the complete model on the building defect verification set reached 53.2, AP50 reached 76.4, AP75 reached 56.3, the average precision of small targets (AP-small) reached 30.8, and the average recall rate (AR100) with a maximum of 100 detection results reached 72.2, which is better than the comparison models such as YOLOv8s, YOLOv11s, RT-DETR-R18 and the original DEIMv2. The improvement of AP75 and AP-small shows that the present invention has a significant improvement on defect localization quality and small-scale defect recall.

[0066] Table 1

[0067] In the ablation embodiment, the detection results are shown in Table 2. The average accuracy of the original detection model is 47.03, which is improved to 51.08 after adopting the residual geometric-semantic fusion structure, and further improved to 53.16 for the complete model. The results of directly splicing depth information and directly injecting geometric features are lower than the baseline, indicating that the residual cross-attention fusion method and channel recalibration mechanism proposed in this invention are necessary and effective.

[0068] Table 2

[0069] With the deployment method using cached geometric features, the comparison results are shown in Table 3. The complete model can still maintain a real-time detection speed of 52.4 frames per second (FPS). In complex scenes such as low light, strong texture backgrounds, motion blur, and small-scale targets, the average accuracy of the complete model is higher than the baseline, making it suitable for real-time or semi-online applications in building inspection, post-disaster assessment, and engineering operation and maintenance. A comparison of the convergence trends of training average accuracy (AP) and small target average accuracy (AP-small) is shown below. Figure 10 and Figure 11 As shown, Figure 10 In the middle, from top to bottom, are the training average accuracy curves of the GDA-DEIMv2 model, the DEIMv2+GFF model, and the DEIMv2 model. Figure 11 In the middle, from top to bottom, are the small target average accuracy curves of the GDA-DEIMv2 model, the DEIMv2+GFF model, and the DEIMv2 model, respectively.

[0070] Table 3

[0071] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A fast detection method for multiple types of building defects using geometric prior embedding, characterized in that, include: Step 1: Collect images of building defects after the earthquake, define the categories of the defect images and label them with rectangular boxes to construct defect detection samples; Step 2: Construct a multi-scale building appearance defect detection backbone network. Input the defect samples into the multi-scale building appearance defect detection backbone network and output multi-scale detection semantic features, which include defect categories and bounding box prediction results. Step 3: Input the defect samples from the multi-scale building appearance defect detection backbone network into the geometric prior extraction branch to obtain geometric prior features; Step 4: Align and fuse the geometric prior features with the multi-scale detection semantic features to obtain the geometric-semantic fusion multi-scale detection semantic features; Step 5: Perform channel recalibration on the multi-scale detection semantic features after geometry-semantic fusion to enhance the channel responses related to the defined defect categories; Step 6: Based on steps 2-5, implement end-to-end training and inference. During the training phase, perform end-to-end optimization of trainable parameters. During the inference phase, disable data augmentation and dense supervision strategies on the training side and output the category, confidence, and bounding box for each building defect instance. Step 7: Based on the actual application requirements, select the corresponding estimation method to generate building inspection results, visualization annotation map and defect statistics, and output the inspection results to the system terminal.

2. The method for rapid detection of multiple types of building defects using geometric prior embedding according to claim 1, characterized in that, Step 1 specifically includes: Step 1.1: Obtain defect images of post-earthquake buildings, concrete components, facades, and interior and exterior structural surfaces using one of the following methods: drone inspection, mobile terminal photography, fixed camera equipment acquisition, or engineering inspection database. The defect images are RGB images. Step 1.2: Identify the defect category in the defect image, where the defect categories include concrete spalling, wall damage, exposed rebar, and rebar buckling; concrete spalling indicates irregular areas formed by the detachment of the protective layer / surface material; wall damage indicates cracks, gaps, and material deterioration on the surface of walls, masonry, or components; exposed rebar indicates slender targets formed by the exposed rebar after the protective layer is damaged; rebar buckling indicates a geometric state in which the centerline of exposed rebar is bent, locally increased in curvature, or laterally offset; Step 1.3: Establish the correspondence between image number, file name, width and height, category number, bounding box coordinates, area and instance number according to the general object detection data format, and uniformly represent the annotation box in the form of [x,y,w,h].

3. The method for rapid detection of multiple types of building defects using geometric prior embedding according to claim 1, characterized in that, The multi-scale building appearance defect detection backbone network in step 2 includes a lightweight visual feature extraction sub-network, a detection scale conversion sub-network, a hybrid coding module, and a decoder connected in sequence. The lightweight visual feature extraction subnetwork is obtained through DINOv3 visual representation distillation and is used to encode images in the input defect detection samples and extract single-scale semantic features. The detection scale conversion subnetwork adopts a spatially tuned adaptation structure, which includes a main detection module, parameter-free bilinear resampling, lightweight convolutional detail branch, and multi-scale fusion structure to simultaneously preserve high-resolution details and deep semantic information. The main detection module includes an upsampling branch, an identity mapping branch, and a downsampling branch. The detection scale conversion subnetwork is used to convert single-scale semantic features into three detection scales: P3, P4, and P5. The hybrid encoding module is used for bidirectional cross-scale feature fusion and selects high-quality candidates as the initial target query. The decoder is used to iteratively refine the initial target query and output multi-scale detection semantic features.

4. The method for rapid detection of multiple types of building defects using geometric prior embedding according to claim 3, characterized in that, Step 2 specifically includes: Step 2.1: Encode the images in the defect detection samples based on a lightweight visual feature extraction network, and extract the multi-scale semantic features of the corresponding images. ; Step 2.2: Multi-scale semantic features Identity mapping is performed to obtain P4 detection scale features. Lightweight convolutional detail branches are used to set up the convolutional structures for upsampling and downsampling branches. The upsampling branch employs two 3D layers. 3 convolutions, with the downsampling branch using two layers of 3 3 convolutions are used to obtain features at three detection scales, P3, P4, and P5, through upsampling, identity mapping, and downsampling branches. Step 2.3: Align the features of the three detection scales P3, P4, and P5 through 1×1 input projection to complete the channel alignment; Step 2.4: Input the channel-aligned features into the hybrid encoding module. The low-resolution deep semantic information is transferred to the high-resolution features through the corresponding scale convolution fusion function and the top-down path. The details and localization information in the high-resolution features are fed back to the low-resolution features through the corresponding scale convolution fusion function and the bottom-up path, thus completing the bidirectional cross-scale fusion and obtaining features E3, E4 and E5 containing deep semantic information as well as defect boundaries, locations and scales. Step 2.5: Select features E3, E4 and E5 as candidates, and select candidates with quality higher than the preset value as the initial target query and transmit them to the decoder. Use multi-scale encoded features as keys and values ​​to iteratively refine the initial target query. The classification prediction head and bounding box regression prediction head output multi-scale detection semantic features. Multi-scale semantic features The calculation formula is: （1）； In formula (1), The input image has dimensions H×W×3. Here is a visual feature extraction network with parameter θb; cb represents the number of feature channels; H and W represent the height and width of the input defect detection image, respectively.

5. The method for rapid detection of multiple types of building defects using geometric prior embedding according to claim 1, characterized in that, Step 3 specifically includes: A pre-trained monocular geometric estimation model is used as the geometric prior extraction branch; wherein, the geometric prior extraction branch keeps the parameters frozen during training and only performs forward inference; the gradient from the detection loss is not backpropagated to the geometric prior extraction branch; The input image is mapped to a pixel-aligned depth map D and a ray map M, where the ray map records the ray origin t and ray direction d corresponding to each pixel; Based on the pixel coordinates, ray origin coordinates, and unit ray direction vector, the three-dimensional spatial point P corresponding to the pixel is recovered by combining the depth map D and the ray map M. Robust normalization is performed on the depth map, and the horizontal and vertical depth gradients are calculated using the difference operator; The surface normal is estimated based on the spatial variation of adjacent three-dimensional points, and the local curvature is calculated based on the variation of the surface normal. The geometric prior features G are obtained by combining the robustly normalized depth ray direction, depth gradient, surface normal, and local curvature.

6. The method for rapid detection of multiple types of building defects using geometric prior embedding according to claim 1, characterized in that, Step 4 specifically includes: For each detection scale of multi-scale detection semantic features, the geometric prior feature G is size-aligned according to the current semantic feature resolution and mapped to the same channel dimension as the semantic feature through 1×1 convolution / linear projection. Using the corresponding scale's detection semantic features as the query and the linearly projected geometric prior features as the key and value, cross-attention is calculated. ; Set learnable residual coefficients By controlling the injection intensity of geometric prior features at each detection scale, cross-attention is added to the original semantic features in a residual manner to complete alignment and fusion, thereby obtaining multi-scale detection semantic features after geometric-semantic fusion. Among them, learnable residual coefficients Initially set to 0 or a small value close to 0, the model approximates the original detector in the early stages of training; as training progresses, the network adaptively adjusts based on the reliability of geometric information. ; Multi-scale detection of semantic features The calculation formula is: （2）； In formula (2), The first before the integration i Scale semantic features.

7. The method for rapid detection of multiple types of architectural defects using geometric prior embedding according to claim 3, characterized in that, Step 5 specifically includes: Multi-scale detection semantic features after geometric-semantic fusion The input hybrid coding module is subjected to cross-scale interaction and fusion. After the fusion output at each scale, a channel recalibration structure is set to select channels for the detection features that have aggregated semantic and localization information. Global average pooling is performed on the fused multi-scale detection semantic features to obtain a channel-level global description. Channel weights are generated through two layers of lightweight mapping and nonlinear activation. Based on channel weight Multi-scale detection semantic features after geometry-semantic fusion Channel-by-channel multiplication recalibration is performed, where the compression ratio of the channel recalibration structure is set according to the number of detection feature channels. Let C be the number of detection feature channels of the fused multi-scale detection semantic features, and r be the compression ratio of the channel recalibration structure, satisfying the following conditions: ,in, The preset baseline number of channels; the two-layer lightweight mapping includes a first lightweight mapping layer and a second lightweight mapping layer. The first lightweight mapping layer reduces the number of channels from... Compress to The second lightweight mapping layer increases the number of channels from... Restore to To generate channel weights corresponding to each channel of the detected features; By recalibrating the channels, we can assess the channel responses related to concrete spalling edges, damaged wall areas, exposed slender rebar structures, and local curvature of rebar buckling, while suppressing redundant channels corresponding to wall textures, shadow boundaries, stains, and edges of non-defective components.

8. The method for rapid detection of multiple types of architectural defects using geometric prior embedding according to claim 3, characterized in that, Step 6 specifically includes: During the training phase, steps 2-5, the spatial tuning and adaptation structure, the geometric projection layer, the cross-attention layer, the residual scaling factor, the channel recalibration structure, the detection decoding structure, the classification prediction head, and the bounding box regression prediction head are used as the trainable parts; the pre-trained geometric prior extraction branch is used as the frozen part. End-to-end optimization is performed using dense one-to-one matching supervision, matching quality-aware classification loss, bounding box regression loss, and auxiliary layer supervision. During the inference phase, the training-side data augmentation and dense supervision strategies are turned off. Based on the input building defect inspection images, multi-scale semantic detection feature extraction, pixel-level depth-ray geometric prior feature extraction, geometric-semantic fusion, channel recalibration, and detection decoding are performed sequentially to output the category, confidence level, and bounding box of each building defect instance.

9. The method for rapid detection of multiple types of architectural defects using geometric prior embedding according to claim 1, characterized in that, Step 7 specifically includes: Based on the actual application requirements, select one of the following methods to generate building defect detection results, visualization annotation maps and defect statistics: geometric feature caching, keyframe geometric estimation, online geometric estimation or lightweight geometric estimation, and output the detection results to the system terminal.

10. A rapid detection system for multi-type building defects using geometric prior embedding, applied to the rapid detection method for multi-type building defects using geometric prior embedding as described in any one of claims 1-9, characterized in that, include: Image input module, used to receive building inspection images; The main detection module is used to generate the multi-scale semantic detection features based on the building inspection images; The geometric prior module is used to generate the pixel-level depth-ray geometric prior features based on the building inspection images; The geometry-semantic fusion module is used to perform the scale alignment and geometry-semantic fusion based on the multi-scale semantic detection features and the pixel-level depth-ray geometric prior features; The channel recalibration module is used to perform the channel recalibration on the fused multi-scale detection features. The training optimization module is used to perform the end-to-end training; The inference deployment module is used to schedule geometric feature caching, keyframe geometric estimation, or online geometric estimation to perform inference based on real-time inspection, edge deployment, or offline analysis requirements. The results output module is used to output the category, confidence level, bounding box coordinates, and visualization results of building defects.

Images