A large building surface crack image segmentation method, system and device

Abstract

The application provides a large building surface crack image segmentation method, system and equipment, comprising: obtaining a building surface original image of a target area to be detected; extracting a multi-dimensional feature vector of the building surface original image through a heterogeneous network composed of a convolutional neural network and a Transformer decoder branch, and generating an image prediction result; dynamically adjusting based on the image prediction result, and constraining by using a consistency supervision mechanism to obtain an optimized prediction label; performing pixel-level contrast learning on the multi-dimensional feature vector and the optimized prediction label to obtain a final crack segmentation result; and performing reasoning based on the final crack segmentation result to obtain a crack segmentation probability map. The application automatically identifies and accurately segments surface cracks of key infrastructures such as dams through an artificial intelligence algorithm, improves crack detection accuracy under the condition of limited labeled data, enhances model robustness, and thus optimizes facility operation and maintenance efficiency and improves structural safety.

Description

Technical Field

[0001] This application belongs to the field of image data processing technology, and relates to a building extraction method, and in particular to a method, system and device for segmenting crack images on the surface of a large building. Background Technology

[0002] Dams, as critical national water infrastructure, play a vital role in water resource regulation, flood control and disaster relief, energy supply, and shipping development. With the increasing age of many dams, performance degradation caused by structural material aging and long-term loads is becoming increasingly apparent. Continuous and effective safety monitoring and operation and maintenance management of dams has become a focus of industry attention. Among these, surface cracks are a common structural defect, and their early detection and accurate assessment are crucial technical aspects of implementing full life-cycle health management of dams. The continued expansion of cracks can affect the overall integrity of the dam body and exacerbate internal seepage risks, posing a potential threat to the long-term safe operation of the dam.

[0003] Currently, my country has built numerous large and medium-sized dams, some of which have been in operation for nearly or beyond their design reference period. In such projects, failure to promptly identify and address structural defects could affect the dam's safety and adversely impact downstream socio-economic activities and the ecological environment.

[0004] Among the many hidden dangers affecting the structural safety of dams, the early identification and dynamic assessment of surface structural cracks are particularly crucial. The formation mechanism of cracks is complex, potentially limited by material aging and temperature stress, or originating from geological subsidence, water pressure impact, and sudden natural disasters. Once cracks develop out of control, they not only weaken the overall structural rigidity but may also become seepage channels, exacerbating the imbalance of internal forces within the dam. Therefore, improving the automation and accuracy of crack detection is the technological foundation for achieving intelligent monitoring of dams throughout their entire lifecycle—from prediction and early warning to pre-judgment and pre-control.

[0005] Currently, the detection of cracks in dams in engineering practice mainly relies on traditional manual visual inspections and localized instrument-assisted detection methods. For example, inspectors use equipment such as laser scanners, infrared thermal imagers, or ultrasonic rangefinders to conduct fixed-point or sampling inspections of the dam surface. However, these existing technologies have the following significant drawbacks:

[0006] (1) Inefficient and costly: Manual inspection requires professional personnel to be on-site, which is labor-intensive and has a long inspection cycle. Moreover, it is difficult to achieve comprehensive and high-frequency coverage for large dam structures, resulting in high labor and time costs.

[0007] (2) Poor real-time performance and strong subjectivity: Manual and instrument-assisted detection are mostly non-continuous and periodic tasks, which cannot monitor the dynamic development of cracks in real time. At the same time, the detection results rely heavily on the experience and subjective judgment of the inspectors, which can easily lead to missed detections and misjudgments, making it difficult to ensure the consistency and objectivity of the data;

[0008] (3) Weak environmental adaptability and insufficient robustness: The surface environment of the dam is complex and changeable, often with interference factors such as uneven lighting, shadow occlusion, surface stains, and low contrast background. Traditional detection methods (including some algorithms based on simple image processing) will have a sharp drop in recognition accuracy and stability when dealing with these complex scenarios, making it difficult to meet the monitoring requirements of all-weather and high reliability.

[0009] In recent years, with the rapid development of artificial intelligence and computer vision technologies, deep learning-based image recognition technology has provided a new solution for the automated detection of dam cracks. In particular, deep learning models such as CNN (Convolutional Neural Network) and Vision Transformer, with their powerful feature extraction and pattern recognition capabilities, have demonstrated performance far exceeding traditional methods in crack segmentation and localization tasks. However, applying these advanced models to practical engineering still faces a core bottleneck:

[0010] Current mainstream deep learning models heavily rely on large-scale, high-quality pixel-level labeled data for supervised training. In the specific application scenario of dam crack detection, obtaining such labeled data is extremely difficult. On the one hand, professional labeling requires significant manpower and time, making it extremely expensive; on the other hand, due to the complex structure of dams and the diverse and uneven distribution of crack morphologies, obtaining a complete dataset covering various working conditions and crack types is virtually impossible. This "data scarcity" problem severely limits the performance and deployment of existing advanced models in practical engineering, resulting in poor model generalization ability and insufficient robustness, making it difficult to meet the urgent needs of smart water conservancy projects for automated, precise, and lightweight deployment.

[0011] In summary, existing technologies suffer from low efficiency, high cost, and poor adaptability in traditional detection methods. While deep learning-based automated detection methods offer superior performance, they are limited by their strong dependence on massive amounts of labeled data. Therefore, providing an intelligent algorithm framework that can achieve high-precision and robust dam crack segmentation even under conditions of extremely scarce labeled data, in order to overcome the aforementioned shortcomings of existing technologies, has become an urgent technical problem to be solved in this field. Summary of the Invention

[0012] The purpose of this application is to provide a method, system and device for image segmentation of cracks on the surface of large buildings, in order to solve the problem of high-precision and high-efficiency automatic identification and segmentation of cracks on the surface of dams under the condition of extremely scarce labeled data in the existing technology, so as to overcome the problems of strong dependence on manual labor, high labeling cost and poor environmental adaptability of traditional detection methods and existing deep learning models.

[0013] In a first aspect, this application provides a method for crack image segmentation of a large building surface, comprising the following steps: acquiring an original image of the building surface of the target area to be detected; extracting a multidimensional feature vector from the original building surface image using a heterogeneous network composed of a convolutional neural network and a Transformer decoder branch, and generating an image prediction result; the multidimensional feature vector includes a feature map of labeled data and a feature map of an unlabeled image; the image prediction result includes a crack segmentation probability feature map and a comparison result; dynamically adjusting based on the image prediction result and constraining it using a consistency supervision mechanism to obtain an optimized prediction label; performing pixel-level comparison learning on the multidimensional feature vector and the optimized prediction label to achieve feature alignment between the labeled data feature map and the unlabeled image, thereby obtaining a final crack segmentation result; and performing inference based on the final crack segmentation result to obtain a crack segmentation probability map, thereby realizing crack detection and evaluation of the building surface.

[0014] In one implementation of the first aspect, the extraction of multidimensional feature vectors from the original building surface image through a heterogeneous network composed of a convolutional neural network and a Transformer decoder branch includes: labeling the original building surface image to obtain a labeled building surface image and an unlabeled building surface image; performing encoding and decoding processing and feature extraction on the labeled building surface image to obtain a feature map with labeled data; and using a multi-scale perturbation strategy to predict the unlabeled image to obtain a multi-scale perturbation prediction result.

[0015] In this implementation, by integrating two different decoder architectures, CNN and Transformer, joint modeling of crack details and global structure is achieved, enhancing the model's multi-scale perception capability and adaptability to complex textures.

[0016] In one implementation of the first aspect, the process of encoding / decoding and feature extraction based on the labeled building surface image to obtain a feature map with labeled data includes: extracting features based on the labeled building surface image to obtain multi-scale features of the labeled image; inputting the image features extracted by the encoder into a CNN decoder, and having the CNN decoder output a pixel-level crack segmentation probability map corresponding to the labeled building surface image; updating parameters based on the pixel-level crack segmentation probability map, and mapping the multi-scale features of the labeled image to generate a feature map with labeled data.

[0017] In one implementation of the first aspect, the multi-scale perturbation strategy includes: a strong perturbation path and a weak perturbation path; using the multi-scale perturbation strategy to predict the unlabeled image and obtain the multi-scale perturbation prediction result includes: performing encoder feature extraction and CNN decoder reconstruction processing on the unlabeled image through a convolutional neural network to obtain a crack segmentation probability map of the unlabeled image and generate a crack segmentation probability feature map; using a Transformer model to extract and decode the feature map of the unlabeled image to obtain a crack segmentation prediction map, which serves as a pseudo-label for the unlabeled image; comparing the pseudo-label with the crack segmentation probability feature map to constrain the training of the model on the unlabeled image and obtain a comparison result; and using the crack segmentation probability feature map and the comparison result as the multi-scale perturbation prediction result.

[0018] In one implementation of the first aspect, dynamic adjustment based on the image prediction result and constraint by a consistency supervision mechanism to obtain the optimized prediction label includes: data integration based on the crack segmentation probability feature map to generate an optimized pseudo-label mask; sampling unlabeled pixels based on the optimized pseudo-label mask to obtain a pixel sample set of unlabeled data; extracting pixels with high confidence prediction from the pixel sample set of unlabeled data, and using the pixels with high confidence prediction as the optimized prediction label.

[0019] In this implementation, a pseudo-label optimization mechanism based on uncertainty estimation is used to address the problem of pseudo-label noise interference and improve the utilization efficiency of unlabeled data in semi-supervised learning. Furthermore, the consistency supervision mechanism is used to improve the robustness of the model to input perturbations, ensuring that the prediction results of the two branches remain consistent under different views, thereby enhancing the semantic alignment capability of the model.

[0020] In one implementation of the first aspect, the process of integrating data based on the crack segmentation probability feature map to generate an optimized pseudo-label mask map includes: obtaining a fused average probability map based on the crack segmentation probability feature map; calculating the prediction entropy value of each pixel in the fused average probability map; and performing dynamic threshold filtering based on the prediction entropy value of each pixel to obtain the optimized pseudo-label mask map.

[0021] In one implementation of the first aspect, sampling unlabeled pixels based on the optimized pseudo-label mask to obtain a pixel sample set of unlabeled data includes: extracting several samples from the optimized pseudo-label mask to construct a pixel sample set of unlabeled data; the pixel sample set of unlabeled data includes positive samples and negative samples; the positive samples are pixel features within the crack area indicated by the mask, and the negative samples are pixel features sampled from the background area outside the mask.

[0022] In one implementation of the first aspect, pixel-level comparative learning is performed on the multidimensional feature vector and the optimized predicted label to achieve feature alignment between the labeled data feature map and the unlabeled image, resulting in the final crack segmentation result. This includes: extracting high-confidence pseudo-label regions based on the multidimensional feature vector and the optimized predicted label; constructing anchor point comparison pairs based on the high-confidence pseudo-label regions; obtaining the similarity between the anchor points and positive and negative sample features based on the anchor point comparison pairs, and calculating pixel-level comparative loss; the pixel-level comparative loss includes: supervision loss, consistency loss, and comparative loss.

[0023] In this implementation, a pixel-level contrastive learning module is used to improve the intra-class cohesion and inter-class separation of the model in the feature space, thereby enhancing the feature differentiation ability between cracks and the background.

[0024] Secondly, this application provides a crack image segmentation system for large building surfaces, comprising: an acquisition module for acquiring original images of the building surface of the target area to be detected; a heterogeneous dual-branch network construction module for extracting multi-dimensional feature vectors from the original building surface image through a heterogeneous network composed of convolutional neural network and Transformer decoder branches, and generating image prediction results; the multi-dimensional feature vectors include: feature maps of labeled data and feature maps of unlabeled images; the image prediction results include: crack segmentation probability feature maps and comparison results; an optimization supervision module for dynamically adjusting based on the image prediction results and using a consistency supervision mechanism for constraint to obtain optimized prediction labels; a pixel-level comparison module for performing pixel-level comparison learning on the multi-dimensional feature vectors and the optimized prediction labels to achieve feature alignment between the labeled data feature maps and the unlabeled images, and obtain the final crack segmentation result; and an output and inference module for inference based on the final crack segmentation result to obtain crack segmentation probability maps, thereby realizing crack detection and evaluation of building surfaces.

[0025] Finally, this application provides an electronic device, including a processor and a memory. The memory is used to store a computer program; the processor is connected to the memory and is used to execute the computer program stored in the memory, so that the large building surface crack image segmentation device performs the large building surface crack image segmentation method.

[0026] As described above, the method, system, and device for segmenting crack images on large building surfaces according to this application have the following beneficial effects:

[0027] (1) The crack image segmentation method for large building surfaces provided in this application constructs a dual-branch heterogeneous network architecture composed of a convolutional neural network and a Transformer decoder. This architecture can process strong perturbation views and weak perturbation views of the same unlabeled image respectively, and can fully explore the multi-scale and multi-morphological features of cracks in the image, thereby improving the model's ability to identify small cracks and irregular edges.

[0028] (2) The dynamic pseudo-label screening and consistency supervision mechanism in this application generates pseudo-labels through the Transformer branch, and then uses the entropy value of the prediction result to estimate the pixel-level confidence, thereby eliminating uncertain regions and retaining only high-confidence regions for training, effectively suppressing pseudo-label noise. At the same time, the consistency loss function between the outputs of the two branches strengthens the semantic consistency learning of the model under different perturbation conditions and improves the stability of the model.

[0029] (3) The pixel-level contrast learning module based on uncertainty guidance introduced in this application promotes the aggregation of similar crack pixels in the feature space and the separation of dissimilar pixels, thereby improving the model's ability to distinguish between cracks and background boundaries.

[0030] (4) The training objective function design method of the joint multi-source supervision signal in this application comprehensively considers three types of loss functions: label supervision, consistency learning and contrastive learning, and jointly optimizes the model from three dimensions: pixel accuracy, semantic stability and feature discriminability. Attached Figure Description

[0031] Figure 1 The diagram shows a hardware application scenario of the method for segmenting crack images on large building surfaces described in this application in one embodiment.

[0032] Figure 2 The diagram shown is a flowchart of an embodiment of the method for segmenting crack images on large building surfaces as described in this application.

[0033] Figure 3 The diagram shows a data processing flow in the method for segmenting crack images on large building surfaces as described in this application.

[0034] Figure 4 The diagram shown is a flowchart of step S2 in the method for segmenting crack images on large building surfaces as described in this application.

[0035] Figure 5 The diagram shown is a flowchart of step S22 in the method for segmenting crack images on large building surfaces as described in this application.

[0036] Figure 6 The diagram shown is a flowchart of step S23 in the method for segmenting crack images on large building surfaces as described in this application.

[0037] Figure 7 The diagram shown is a flowchart of step S3 in the method for segmenting crack images on large building surfaces as described in this application.

[0038] Figure 8 The diagram shown is a flowchart of step S31 in the method for segmenting crack images on large building surfaces as described in this application.

[0039] Figure 9 The diagram shown is a flowchart of step S4 in the method for segmenting crack images on large building surfaces as described in this application.

[0040] Figure 10 The diagram shown is a CPCL structure diagram of an embodiment of the large building surface crack image segmentation method described in this application.

[0041] Figure 11The diagram shows experimental results of an embodiment of the method for segmenting crack images on the surface of a large building as described in this application.

[0042] Figure 12 The diagram shown is a schematic representation of the principle structure of the large building surface crack image segmentation system described in this application in one embodiment.

[0043] Figure 13 The diagram shown is a schematic representation of the electronic device described in this application.

[0044] Component designation explanation

[0045] 11 Data acquisition module 12 Data processing module 13 Dual-branch heterogeneous module 1301 Dual-branch heterogeneous network structure module 1302 Multi-scale perturbation design module 14 Consistency Learning Module 15 Comparative learning module 121 Get Module 122 Heterogeneous dual-branch network building module 123 Optimize the supervision module 124 Pixel-level contrast module 125 Output and Inference Module 131 processor 132 memory Detailed Implementation

[0046] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0047] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0048] The following embodiments of this application provide a method for segmenting crack images on the surface of large buildings, which solves the problem of high-precision and high-efficiency automatic identification and segmentation of cracks on the surface of dams under the condition of extremely scarce labeled data in the prior art. This overcomes the problems of traditional detection methods and existing deep learning models, such as strong dependence on manual labor, high labeling costs, and poor environmental adaptability.

[0049] like Figure 1 As shown, the hardware application scenario of the crack image segmentation method for large building surfaces described in this application is illustrated in one embodiment. Specifically, it includes: a data acquisition module 11, a data processing module 12, a dual-branch heterogeneous module 13, a consistency learning module 14, and a contrastive learning module 15.

[0050] The data acquisition module 11 acquires the original image of the building surface of the target area to be detected. The data processing module 12 processes the original image of the building surface of the target area to be detected and extracts key features from the image. The dual-branch heterogeneous module 13 includes a dual-branch heterogeneous network structure module 1301 and a multi-scale perturbation design module 1302, which is used to construct a dual-branch architecture composed of a convolutional neural network (CNN) and a Transformer decoder, respectively processing the strongly perturbated view and the weakly perturbated view of the same unlabeled image. Differentiated data augmentation strategies (such as random cropping, color perturbation, CutMix, etc.) are used to simulate the changes of the image in the real environment to achieve multi-view feature expression. The two branches share an encoder but use heterogeneous decoders, thereby improving the robustness and feature extraction ability of the model while maintaining structural semantic stability. The consistency learning module 14 uses the pseudo-labels generated by the Transformer branch and combines them with the entropy value of the predicted distribution to measure uncertainty and dynamically select high-confidence regions as pseudo-supervision signals. In the early stage of training, a strict confidence threshold is used to control noise interference, and in the later stage, the threshold is gradually relaxed to expand the supervision range. Simultaneously, a cross-branch consistency loss function is designed to ensure semantic consistency between the output predictions of the CNN branch and the Transformer branch under different perturbation conditions, thereby enhancing the model's representation stability of crack structures. The contrastive learning module 15 is used to introduce a contrastive learning module into the CNN branch. It samples positive and negative samples from the pixel-level embeddings of high-confidence pseudo-label regions, constructs an InfoNCE-based contrastive loss function, promotes the clustering of similar pixels in the feature space and the separation of dissimilar pixels, improving the ability to distinguish cracks from the background. To improve computational efficiency, a dynamically updated feature memory queue is introduced to store prototype embedding vectors, reducing positive and negative sample selection errors while maintaining training stability.

[0051] In summary, this application achieves high-precision crack identification under the condition of limited labeled samples, reducing the cost of manual labeling; it has strong robustness and good boundary detail perception capabilities, adapting to complex textures and low-contrast images; it can be deployed in the inspection system of large structures such as dams, bridges, and tunnels, contributing to the intelligent management of infrastructure.

[0052] The method for segmenting crack images on large building surfaces provided in this application will be described in detail below with reference to the accompanying drawings in the embodiments of this application.

[0053] Please see Figure 2 and Figure 3 The diagrams shown are a flowchart illustrating an embodiment of the method for segmenting crack images on large building surfaces according to this application, and a data processing flowchart illustrating the method for segmenting crack images on large building surfaces according to this application. Figure 2 and Figure 3As shown in the figure, this embodiment provides a method for segmenting crack images on the surface of a large building.

[0054] The proposed method for segmenting crack images on large building surfaces can achieve high-precision crack identification under limited labeled samples, reducing the cost of manual labeling; it has strong robustness and good boundary detail perception capabilities, adapting to complex textures and low-contrast images; it can be deployed in inspection systems for large structures such as dams, bridges, and tunnels, contributing to the intelligent management of infrastructure.

[0055] The method for segmenting crack images on the surface of large buildings specifically includes the following steps:

[0056] S1, acquire the original image of the building surface of the target area to be detected.

[0057] In this embodiment, the building facades within the target area to be detected are divided into several manageable areas (e.g., by floor or by facade), and each area is numbered to ensure no omissions, thus performing refined building area division and planning. Then, appropriate equipment is selected based on the plan, and standardized image acquisition is performed. Next, the massive amount of acquired images is systematically processed and labeled.

[0058] Specifically, firstly, based on the detection target, the building facade to be inspected is systematically divided into several manageable grids or areas (such as by floor, orientation, or structural unit), and each area is assigned a unique identifier. This step aims to ensure that subsequent data collection is thorough and comprehensive, while also developing detailed safety plans and data collection routes.

[0059] Then, select appropriate equipment according to the plan and perform standardized image acquisition. Considering building height and accessibility, select equipment such as drones or high-definition cameras, and under uniform lighting conditions, take pictures along a predetermined path with the lens perpendicular to the building surface. During acquisition, ensure high overlap between adjacent images, and take multi-scale close-up shots, including scale bars, of suspected defect areas to obtain high-resolution, distortion-free original images.

[0060] In other words, the target area and shooting angle can be determined according to the detection requirements. Then, high-resolution cameras, drones, mobile phones and other image acquisition devices can be used to take multi-angle, full-coverage pictures of the building surface under suitable lighting conditions (such as daytime without strong direct sunlight or uniform artificial light source) to ensure that the images are clear, unobstructed and without obvious distortion. The acquired images are stored in the original format (such as RAW or high-resolution JPEG) for subsequent image processing, defect identification or 3D modeling and other analysis work.

[0061] S2, using a heterogeneous network composed of a convolutional neural network and a Transformer decoder branch, extracts multi-dimensional feature vectors from the original image of the building surface and generates image prediction results. The multi-dimensional feature vectors include feature maps of labeled data and feature maps of unlabeled images; the image prediction results include crack segmentation probability feature maps and comparison results.

[0062] In this embodiment, a dual-branch heterogeneous network structure and multi-scale perturbation design are employed. Specifically, a dual-branch architecture consisting of a convolutional neural network and a Transformer decoder is constructed to process strongly perturbated and weakly perturbated views of the same unlabeled image, respectively. Differentiated data augmentation strategies (such as random cropping, color perturbation, and CutMix) simulate changes in the image under real-world conditions, achieving multi-view feature representation. The two branches share an encoder but employ heterogeneous decoders, thereby improving the model's robustness and feature extraction capabilities while maintaining structural semantic stability.

[0063] Please see Figure 4 The diagram shows a flowchart of step S2 in the method for segmenting crack images on large building surfaces described in this application. Figure 4 As shown, step S2 includes the following steps:

[0064] S21, based on the original image of the building surface, annotate it to obtain an annotated image and an unannotated image of the building surface.

[0065] In this embodiment, the collected original images of building surfaces are systematically processed and labeled. After backing up the image data, it is categorized according to the divided regions, and the files are renamed using standardized rules. In the data labeling stage, key information tags, such as the region number, shooting time, and orientation, are added to each image to form a well-structured and complete original image dataset, laying a solid foundation for subsequent detection and analysis.

[0066] S22, based on the labeled image of the building surface, perform encoding / decoding processing and feature extraction to obtain a feature map with labeled data. Please refer to [link to relevant documentation]. Figure 5 The diagram shows a flowchart of step S22 in the method for segmenting crack images on large building surfaces described in this application. Figure 5 As shown, step S22 includes the following steps:

[0067] S221, Based on the marked image of the building surface, feature extraction is performed to obtain multi-scale features of the marked image;

[0068] S222, The image features extracted by the encoder are input into the CNN decoder, and the CNN decoder outputs a pixel-level crack segmentation probability map corresponding to the marked image of the building surface;

[0069] S223, update the parameters based on the pixel-level crack segmentation probability map, and map the multi-scale features of the labeled image to generate a feature map with labeled data.

[0070] In this embodiment, image semantics are modeled based on a heterogeneous dual-branch structure. The backbone encoder adopts a shared CNN structure, connected to a convolutional decoder and a Transformer decoder respectively. The Transformer decoder processes weakly perturbed images to maintain structural integrity; the CNN decoder processes strongly perturbed images to enhance detail robustness.

[0071] Specifically, training images with real crack annotations are input into the encoder network of a CSCL (Computer Supported Collaborative Learning) model to extract multi-scale features. These encoder features are then fed into a CNN decoder, which outputs a corresponding crack segmentation probability map. A supervised loss is calculated based on this probability map and the image's real annotations, used to update the model parameters in a supervised manner. Simultaneously, the extracted features are mapped using a projector connected to the encoder to generate a feature map with labeled data, providing a foundation for subsequent pixel-level contrastive learning.

[0072] S23, a multi-scale perturbation strategy is used to predict the unlabeled image, obtaining a multi-scale perturbation prediction result. The multi-scale perturbation strategy includes: a strong perturbation path and a weak perturbation path. Please refer to [link / reference]. Figure 6 The diagram shows a flowchart of step S23 in the method for segmenting crack images on large building surfaces described in this application. Figure 6 As shown, step S23 includes the following steps:

[0073] S231, The unlabeled image is processed by encoder feature extraction and CNN decoder reconstruction through convolutional neural network to obtain the crack segmentation probability map of the unlabeled image and generate crack segmentation probability feature map.

[0074] S232, The feature map of the unlabeled image is extracted and decoded by the Transformer model to obtain the crack segmentation prediction map, which serves as the pseudo label of the unlabeled image;

[0075] S233, Based on the comparison between the pseudo-label and the crack segmentation probability feature map, the training of the model on unlabeled images is constrained to obtain the comparison results;

[0076] S234, the crack segmentation probability feature map and the comparison result are used as the multi-scale perturbation prediction result.

[0077] In this embodiment, two views are generated from the input image: a weakly perturbated image for pseudo-label generation and a strongly perturbated image for feature diversity learning. Weak perturbations include operations such as random scaling, cropping, and flipping, while strong perturbations include color jitter, Gaussian blur, and CutMix.

[0078] Specifically, the unlabeled image is simultaneously input into two parallel processing paths: one with strong perturbation and the other with weak perturbation. The strong perturbation path is a CNN branch (with the same structure as the labeled data branch in step S2), which, after passing through an encoder and a CNN decoder, obtains the crack segmentation probability map of the unlabeled image and generates the corresponding feature map through a projector. The weak perturbation path is a Transformer branch, which applies weaker data augmentation to the same unlabeled image and then inputs it into the Transformer model to extract features and generate a crack segmentation prediction map, which serves as a pseudo-label for the unlabeled image.

[0079] Next, the pseudo-labels generated by the weakly perturbated path are compared with the segmentation probability maps output by the strongly perturbated path, and the consistency loss is calculated to constrain the model's training on unlabeled data. Meanwhile, the outputs of the strongly and weakly perturbated paths provide the foundation for the subsequent pseudo-label fusion.

[0080] It should be noted that this application constructs a dual-branch heterogeneous network architecture consisting of a convolutional neural network (CNN) and a Transformer decoder. This architecture can process strongly perturbed and weakly perturbed views of the same unlabeled image separately. The CNN branch focuses on processing local details after image enhancement (such as color perturbation, Gaussian blur, etc.), while the Transformer branch preserves the global semantic information of the image structure (such as minor perturbations like flipping and cropping). The two complement each other, fully exploring the multi-scale and multi-morphological features of cracks in the image, and improving the model's ability to recognize tiny cracks and irregular edges.

[0081] S3, based on the image prediction results, dynamic adjustments are made, and a consistency supervision mechanism is used for constraint to obtain the optimized predicted label. Please refer to [link / reference]. Figure 7 The diagram shows a flowchart of step S3 in the method for segmenting crack images on large building surfaces described in this application. Figure 7 As shown, step S3 includes the following steps:

[0082] S31, Based on the crack segmentation probability feature map, data integration is performed to generate an optimized pseudo-label mask map. Please refer to [link / reference]. Figure 8 The diagram shows a flowchart of step S31 in the method for segmenting crack images on large building surfaces described in this application. Figure 8 As shown, step S31 includes the following steps:

[0083] S311, Obtain the fused average probability map based on the crack segmentation probability feature map;

[0084] S312, Calculate the prediction entropy value of each pixel in the fused average probability map;

[0085] S313, Dynamic threshold filtering is performed based on the predicted entropy value of each pixel to obtain an optimized pseudo-label mask.

[0086] In this embodiment, it is preferable to use the prediction results of fusion of strong and weak perturbation paths to generate optimized pseudo-labels.

[0087] The Transformer branch is used to predict weakly perturbated images, outputting a class probability map. The prediction entropy value for each pixel is calculated as an uncertainty indicator; the lower the entropy value, the more reliable the prediction. Simultaneously, the initial confidence threshold set in this application retains only pixels with entropy values ​​below the threshold as valid pseudo-labels for training. As training progresses, the threshold is gradually lowered, allowing the model to utilize more high-confidence pseudo-label regions and expand the pseudo-supervision coverage.

[0088] Specifically, the crack segmentation probability maps output by the strongly perturbed CNN branch and the weakly perturbed Transformer branch in the aforementioned steps are averaged pixel-wise to obtain a fused average probability map. The entropy distribution of this average probability map is calculated to measure the uncertainty of each pixel prediction; low-entropy (high-confidence) pixel regions are selected based on a preset entropy threshold, and an optimized pseudo-label mask is generated, retaining only crack pixels predicted by the model with high confidence. The optimized pseudo-label mask improves the accuracy of pseudo-labels for unlabeled data, providing a basis for subsequent pixel sample sampling.

[0089] S32, perform unlabeled pixel sampling based on the optimized pseudo-label mask to obtain a pixel sample set of unlabeled data;

[0090] S33, extract the pixels with high confidence prediction from the pixel sample set of the unlabeled data, and use the pixels with high confidence prediction as the optimized prediction label.

[0091] In this embodiment, several samples are extracted from the optimized pseudo-label mask to construct a pixel sample set of unlabeled data; the pixel sample set of unlabeled data includes positive samples and negative samples; the positive samples are pixel features within the crack area indicated by the mask, and the negative samples are pixel features sampled from the background area outside the mask.

[0092] Based on the optimized pseudo-label mask obtained from the preceding steps, high-confidence pixel samples are selected from the feature map of unlabeled data for comparative learning.

[0093] This application uses cross-entropy loss to constrain the outputs of the CNN branch and the Transformer branch on the same unlabeled sample. With only partial pseudo-labels available, consistency supervision provides the model with stable, low-bias training signals.

[0094] Specifically, a certain number of pixel features are randomly sampled as positive samples within the crack region indicated by the mask, and pixel features are sampled as negative samples from the background region outside the mask, forming a set of pixel samples without labeling data. By only collecting pixels predicted with high confidence by the model, reliable pseudo-label samples are ensured to be considered during the contrastive learning stage. The selected pixel samples will be used in the next pixel-level contrastive learning process.

[0095] It should be noted that this application employs a dynamic pseudo-label selection and consistency supervision mechanism. After generating pseudo-labels through the Transformer branch, the entropy value of the prediction result is used to estimate the pixel-level confidence, thereby eliminating uncertain regions and retaining only high-confidence regions for training, effectively suppressing pseudo-label noise. As training progresses, the confidence threshold gradually decreases, allowing more pseudo-label regions to be included in the supervision scope, realizing the transition of the model from "robust learning" to "broad generalization". Simultaneously, the consistency loss function between the two-branch outputs strengthens the model's semantic consistency learning under different perturbation conditions, improving model stability.

[0096] S4, perform pixel-level comparative learning on the multidimensional feature vector and the optimized predicted label to achieve feature alignment between the labeled data feature map and the unlabeled image, obtaining the final crack segmentation result. (See also...) Figure 9 The diagram shows a flowchart of step S4 in the method for segmenting crack images on large building surfaces described in this application. Figure 9 As shown, step S4 includes the following steps:

[0097] S41, based on the multidimensional feature vector and the optimized predicted label, extract the high-confidence pseudo-label region;

[0098] S42, construct anchor point comparison pairs based on the high-confidence pseudo-label regions;

[0099] S43, obtain the similarity between the anchor point and the positive sample features and the negative sample features according to the anchor point comparison pair, and calculate the pixel-level comparison loss; the pixel-level comparison loss includes: supervision loss, consistency loss and comparison loss.

[0100] Please see Figure 10 The diagram shown is a CPCL structure diagram of an embodiment of the large building surface crack image segmentation method described in this application.

[0101] In this embodiment, the feature maps of the labeled data, the feature maps of the unlabeled images, and the sampled pixel samples and their pseudo-labels obtained in the aforementioned steps are input together into the pixel-level contrastive learning (CPCL) module. Specifically, pixel features with real labels in the labeled data feature map are used as anchor points; unlabeled pixel features of their corresponding category (crack or background) are considered positive samples, while unlabeled pixel features not belonging to that category are considered negative samples.

[0102] The CPCL module measures the similarity between anchor point features and positive and negative sample features, and calculates pixel-level contrastive loss: increasing the similarity between anchor point and positive sample features, and decreasing the similarity between anchor point and negative sample features. Under this contrastive learning constraint, feature alignment between labeled and unlabeled data is achieved, enhancing the model's ability to distinguish cracks from background pixels. The contrastive loss, along with the supervision loss and consistency loss, works together to further optimize model parameters, laying the foundation for the final high-precision crack segmentation output.

[0103] Specifically, the process involves embedding feature extraction and category labeling. Low-dimensional feature representations are extracted from each pixel within the CNN branch and mapped to pseudo-label categories. Only pixel embeddings from high-confidence pseudo-label regions are used in contrastive learning to avoid noise interference.

[0104] Then, positive and negative sample pairs are constructed. Pixels of the same class are used as positive samples, and pixels of different classes are used as negative samples to construct anchor point contrast pairs. The contrastive loss of each anchor point is calculated using the InfoNCE loss function to guide the model in learning a discriminative embedding representation.

[0105] Next, a feature memory queue is maintained. A sliding update prototype queue is used to store historical embedding vectors, mitigating the inter-batch sample offset problem. The feature queue improves sample diversity and training stability while reducing memory computation costs.

[0106] The joint loss function optimization strategy integrates multiple supervision signals into a unified optimization objective, effectively coordinating the training weights among label supervision, consistency learning, and contrastive learning.

[0107] The total loss function consists of: supervision loss, consistency loss, and comparison loss.

[0108] The formula for calculating the pixel-level contrast loss is:

[0109]

[0110] in, Indicates the monitoring loss value; This represents the consistency loss value; This represents the value of the contrast loss function; , All of these represent loss weight parameters, which are dynamically adjusted according to the training phase, with an emphasis on consistency in the early stages and a focus on comparative learning in the later stages.

[0111] In other words, this application comprehensively considers three types of loss functions: label supervision, consistency learning, and contrastive learning, and jointly optimizes the model from three dimensions: pixel accuracy, semantic stability, and feature discriminability.

[0112] It should be noted that this application introduces a pixel-level contrastive learning module based on uncertainty guidance. By selecting high-confidence regions in the pseudo-labels as anchor points, positive and negative sample pairs are constructed, and InfoNCE loss is used for feature embedding optimization. This module promotes the clustering of similar crack pixels in the feature space and the separation of dissimilar pixels, thereby improving the model's ability to distinguish cracks from background boundaries. To reduce computational redundancy during training, this invention further designs a prototype feature memory queue mechanism to dynamically maintain the embedding vectors of high-quality samples, achieving a dual optimization of training efficiency and feature generalization ability.

[0113] S5. Based on the final crack segmentation result, reasoning is performed to obtain a crack segmentation probability map, so as to realize crack detection and evaluation on the building surface.

[0114] In this embodiment, after the above training process, the model of the present invention finally uses the output of the CNN decoder as the final result of crack segmentation. During the inference phase, the image to be detected is input into the trained encoder and CNN decoder to directly obtain the crack segmentation probability map. The segmentation result output by the CNN decoder is used for actual crack detection and evaluation, thus completing the entire CSCL crack segmentation process.

[0115] Specifically, the inference process simplifies the model structure to improve deployment efficiency, facilitating real-time monitoring tasks on edge devices or engineering systems. Only the CNN branch is retained to generate the final crack segmentation results, reducing model complexity and inference latency. The CNN branch, as the main output path, offers advantages such as low computational overhead and simple deployment, making it suitable for real-time detection scenarios in engineering sites.

[0116] It should be noted that this strategy can achieve rapid and low-resource-consumption engineering deployment without sacrificing accuracy.

[0117] The following example uses a set of experimental data to illustrate this.

[0118] Please see Figure 11 The image shown is a schematic diagram of the experimental results of the method for segmenting crack images on the surface of a large building as described in this application in one embodiment.

[0119] The model performance verification here uses a real crack image dataset to comprehensively evaluate the method of this invention, verifying its stability and superiority under different amounts of labeled data. The main contents include the following:

[0120] (1) Dataset partitioning and annotation settings

[0121] Using the CRACK500 dataset as a benchmark, we constructed semi-supervised scenarios with three annotation ratios: 5%, 12.5%, and 25%. We then compared the performance of supervised learning models with mainstream semi-supervised methods, including MT (Mean Teacher), UAMT (Uncertainty-aware meantteacher), CPS (Cross Pseudo Supervision), UGPCL (Uncertainty-Guided Pixel-wise Contrastive Learning), CSCL (Crack Segmentation on via Contrastive Learning), and TCSM_V2 (Transformation-consistent self-ensembling).

[0122] (2) Comparison of experimental results

[0123] like Figure 11 As shown, the qualitative visualization comparison on the CRACK500 dataset is as follows: from left to right, the input image, the labeled image (Ground Truth), and the segmentation results of five methods (CSCL, UAMT, CPS, MT, TCSM_V2) are presented. Figure 11 As can be seen, all the above methods can generally find the crack area, but their performances differ. For example, MT and TCSM_V2 are prone to "breaks / discontinuities" on fine and low-contrast cracks; CPS mask is smoother, but occasionally misidentifies background textures as cracks; UAMT tends to over-segment weak structures; in comparison, CSCL has clearer boundaries and better connectivity.

[0124] As shown above, the method of this invention performs excellently across various metrics (mIoU, mAP, Accuracy), with a particularly significant improvement under low-annotation conditions. Experiments demonstrate that this method has good segmentation capabilities for fine cracks, blurred edges, and low-contrast regions.

[0125] The crack segmentation method for large building surfaces provided in this application achieves high-precision and robust crack segmentation capabilities in low-annotation scenarios by constructing a heterogeneous network structure, introducing a pseudo-label filtering mechanism, and designing consistency and contrastive learning modules. It has significant engineering practical value and promising prospects for widespread application, and is suitable for intelligent inspection and structural health assessment scenarios of various infrastructures. The method adopted in this application is applicable to the automated identification of cracks on the surface of concrete structures such as dams, bridges, and tunnels. It maintains good crack detection performance and engineering practicality even under conditions of scarce labeled data; and demonstrates excellent performance and deployment adaptability in engineering scenarios with limited labeled samples, possessing significant engineering application value and promising prospects for widespread application.

[0126] The scope of protection for the image segmentation method for cracks on the surface of large buildings described in this application is not limited to the order of steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0127] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following... Figure 1 The method for segmenting crack images on the surface of large buildings.

[0128] At any possible level of technical detail, this application can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this application.

[0129] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, (but not limited to) electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0130] The computer-readable program described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network and forwards these instructions to a computer-readable storage medium in the respective computing / processing device. The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as "C" or similar programming languages. Computer-readable program instructions may execute entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of this application.

[0131] This application also provides a crack image segmentation system for large building surfaces. The crack image segmentation system for large building surfaces can implement the crack image segmentation method for large building surfaces described in this application. However, the implementation device of the crack image segmentation method for large building surfaces described in this application includes, but is not limited to, the structure of the crack image segmentation system for large building surfaces listed in this embodiment. All structural modifications and substitutions of the prior art made based on the principles of this application are included within the protection scope of this application.

[0132] The following will describe in detail the crack image segmentation system for large building surfaces provided in this embodiment, with reference to the illustrations.

[0133] This embodiment provides a crack image segmentation system for large building surfaces, including:

[0134] Please see Figure 12 The image shown is a schematic diagram illustrating the principle structure of the large building surface crack image segmentation system described in this application in one embodiment. Figure 12 As shown, the large building surface crack image segmentation system includes: an acquisition module 121, a heterogeneous dual-branch network construction module 122, an optimization supervision module 123, a pixel-level comparison module 124, and an output and inference module 125.

[0135] The acquisition module 121 is used to acquire the original image of the building surface of the target area to be detected.

[0136] In this embodiment, the building facades within the target area to be detected are divided into several manageable areas (e.g., by floor or by facade), and each area is numbered to ensure no omissions, thus performing refined building area division and planning. Then, appropriate equipment is selected based on the plan, and standardized image acquisition is performed. Next, the massive amount of acquired images is systematically processed and labeled.

[0137] The heterogeneous dual-branch network construction module 122 is used to extract multi-dimensional feature vectors from the original image of the building surface through a heterogeneous network composed of a convolutional neural network and a Transformer decoder branch, and generate image prediction results; the multi-dimensional feature vectors include: feature maps with labeled data and feature maps of unlabeled images; the image prediction results include: crack segmentation probability feature maps and comparison results.

[0138] In this embodiment, a dual-branch heterogeneous network structure and multi-scale perturbation design are employed. Specifically, a dual-branch architecture consisting of a convolutional neural network and a Transformer decoder is constructed to process strongly perturbated and weakly perturbated views of the same unlabeled image, respectively. Differentiated data augmentation strategies (such as random cropping, color perturbation, and CutMix) simulate changes in the image under real-world conditions, achieving multi-view feature representation. The two branches share an encoder but employ heterogeneous decoders, thereby improving the model's robustness and feature extraction capabilities while maintaining structural semantic stability.

[0139] Specifically, the original image of the building surface is labeled to obtain a labeled image and an unlabeled image of the building surface; the labeled image of the building surface is encoded and decoded and feature extracted to obtain a feature map of labeled data; a multi-scale perturbation strategy is used to predict the unlabeled image to obtain a multi-scale perturbation prediction result.

[0140] The optimization supervision module 123 is used to dynamically adjust based on the image prediction results and to use a consistency supervision mechanism for constraint to obtain the optimized prediction label.

[0141] In this embodiment, data integration is performed based on the crack segmentation probability feature map to generate an optimized pseudo-label mask map; unlabeled pixel sampling is performed based on the optimized pseudo-label mask map to obtain a pixel sample set of unlabeled data; high-confidence predicted pixels are extracted from the pixel sample set of unlabeled data, and the high-confidence predicted pixels are used as the optimized prediction labels.

[0142] The pixel-level comparison module 124 is used to perform pixel-level comparison learning on the multidimensional feature vector and the optimized predicted label to achieve feature alignment between the labeled data feature map and the unlabeled image, and obtain the final crack segmentation result.

[0143] In this embodiment, a high-confidence pseudo-label region is extracted based on the multi-dimensional feature vector and the optimized predicted label; an anchor point comparison pair is constructed based on the high-confidence pseudo-label region; the similarity between the anchor point and the positive sample features and the negative sample features is obtained based on the anchor point comparison pair, and pixel-level comparison loss is calculated; the pixel-level comparison loss includes: supervision loss, consistency loss and comparison loss.

[0144] Specifically, the feature maps of the labeled data, the feature maps of the unlabeled images, and the sampled pixel samples and their pseudo-labels obtained in the aforementioned steps are input into the pixel-level contrastive learning module. Pixel features with real labels in the labeled data feature maps are used as anchor points; unlabeled pixel features of their corresponding categories (cracks or background) are considered positive samples, while unlabeled pixel features not belonging to that category are considered negative samples.

[0145] The CPCL module measures the similarity between anchor point features and positive and negative sample features, and calculates pixel-level contrastive loss: increasing the similarity between anchor point and positive sample features, and decreasing the similarity between anchor point and negative sample features. Under this contrastive learning constraint, feature alignment between labeled and unlabeled data is achieved, enhancing the model's ability to distinguish cracks from background pixels. The contrastive loss, along with the supervision loss and consistency loss, works together to further optimize model parameters, laying the foundation for the final high-precision crack segmentation output.

[0146] The output and inference module 125 is used to perform inference based on the final crack segmentation result to obtain a crack segmentation probability map, so as to realize crack detection and evaluation on the building surface.

[0147] In this embodiment, after the above training process, the model of the present invention finally uses the output of the CNN decoder as the final result of crack segmentation. During the inference phase, the image to be detected is input into the trained encoder and CNN decoder to directly obtain the crack segmentation probability map. The segmentation result output by the CNN decoder is used for actual crack detection and evaluation, thus completing the entire CSCL crack segmentation process.

[0148] In summary, this application integrates bi-branch consistency learning, pseudo-label optimization strategy, and uncertainty-guided contrast loss design to achieve high-precision and high-robust crack detection capabilities while ensuring low annotation costs.

[0149] This paper presents a segmentation model for crack images on large building surfaces. This system effectively improves segmentation performance and robustness to practical deployment, even with extremely limited labeled crack image data. It is particularly suitable for crack monitoring tasks in large infrastructure such as dams, bridges, and tunnels. Furthermore, this application integrates bi-branch consistency learning, pseudo-label optimization strategies, and uncertainty-guided contrastive loss design, achieving high-precision and robust crack detection capabilities while maintaining low annotation costs.

[0150] It should be noted that the division of the various modules in the above system is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software through processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, module x can be a separate processing element, or it can be integrated into a chip within the system. Alternatively, it can be stored as program code in the system's memory, and its function can be called and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element mentioned here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0151] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more digital signal processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0152] Please see Figure 13 The diagram shows a schematic representation of the electronic device described in this application. Figure 13 As shown, the electronic device includes: a processor 131 and a memory 132; the memory 132 is used to store computer programs; the processor 131 is connected to the memory 132 and is used to execute the computer programs stored in the memory 132 so that the large building surface crack image segmentation device performs the various steps of the large building surface crack image segmentation method as described above.

[0153] Preferably, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0154] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0155] In summary, the method, system, and device for segmenting crack images on large building surfaces provided in this application have the following beneficial effects:

[0156] The crack image segmentation method for large building surfaces provided in this application achieves high-precision and robust crack segmentation capabilities in low-label scenarios by constructing a heterogeneous network structure, introducing a pseudo-label screening mechanism, and designing consistency and contrast learning modules. It has significant engineering practical value and promotion prospects, and is suitable for intelligent inspection and structural health assessment scenarios of various infrastructures.

[0157] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for segmenting crack images on the surface of a large building, characterized in that, include: Acquire the original image of the building surface of the target area to be detected; Annotation is performed on the original image of the building surface to obtain an annotated image and an unannotated image of the building surface; A heterogeneous network consisting of a convolutional neural network and a Transformer decoder branch is used to extract multidimensional feature vectors from the labeled and unlabeled images of the building surface, and to generate image prediction results. The multidimensional feature vector includes: feature maps of labeled data and feature maps of unlabeled images; the image prediction result includes: crack segmentation probability feature map and comparison result; wherein... The annotation image of the building surface is processed by encoding and decoding and feature extraction to obtain a feature map with annotation data; specifically, it includes: extracting multi-scale features of the annotation image of the building surface; inputting the image features extracted by the encoder into the CNN decoder and outputting a pixel-level crack segmentation probability map corresponding to the annotation image of the building surface; updating parameters based on the pixel-level crack segmentation probability map and mapping the multi-scale features of the annotation image to generate a feature map with annotation data. A multi-scale perturbation strategy is applied to the unlabeled image for prediction, resulting in a multi-scale perturbation prediction result. This strategy includes strong perturbation paths and weak perturbation paths. Specifically, it involves: using a convolutional neural network to extract encoder features and a CNN decoder to reconstruct the unlabeled image, obtaining a crack segmentation probability map and generating a crack segmentation probability feature map; using a Transformer model to extract and decode features from the feature map of the unlabeled image, obtaining a crack segmentation prediction map, which serves as a pseudo-label for the unlabeled image; comparing the pseudo-label with the crack segmentation probability feature map to constrain the model's training on the unlabeled image, obtaining a comparison result; and using the crack segmentation probability feature map and the comparison result as the multi-scale perturbation prediction result. The image prediction results are dynamically adjusted and constrained by a consistency supervision mechanism to obtain optimized prediction labels. This includes: integrating data based on the crack segmentation probability feature map to generate an optimized pseudo-label mask; sampling unlabeled pixels based on the optimized pseudo-label mask to obtain a pixel sample set of unlabeled data; extracting high-confidence predicted pixels from the pixel sample set of unlabeled data, and using the high-confidence predicted pixels as the optimized prediction labels. The multidimensional feature vector and the optimized predicted label are compared and learned at the pixel level to achieve feature alignment between the labeled data feature map and the unlabeled image, so as to obtain the final crack segmentation result. Based on the final crack segmentation results, inference is performed to obtain a crack segmentation probability map, so as to realize the detection and evaluation of cracks on the building surface.

2. The method for segmenting crack images on the surface of large buildings according to claim 1, characterized in that, Based on the crack segmentation probability feature map, data integration is performed to generate an optimized pseudo-label mask map, including: The average probability map of the fusion is obtained based on the crack segmentation probability feature map; Calculate the prediction entropy value for each pixel in the fused average probability map; Dynamic threshold filtering is performed based on the predicted entropy value of each pixel to obtain an optimized pseudo-label mask.

3. The method for segmenting crack images on the surface of large buildings according to claim 1, characterized in that, Based on the optimized pseudo-label mask image, unlabeled pixel sampling is performed to obtain a pixel sample set of unlabeled data, including: Several samples are extracted from the optimized pseudo-label mask to construct a pixel sample set of unlabeled data; the pixel sample set of unlabeled data includes positive samples and negative samples; The positive samples are pixel features within the crack area indicated by the mask, and the negative samples are pixel features sampled from the background area outside the mask.

4. The method for segmenting crack images on the surface of large buildings according to claim 1, characterized in that, The multidimensional feature vector and the optimized predicted label are compared and learned at the pixel level to achieve feature alignment between the labeled data feature map and the unlabeled image, resulting in the final crack segmentation result, including: Based on the multidimensional feature vector and the optimized predicted label, the high-confidence pseudo-label region is extracted; Anchor point comparison pairs are constructed based on the high-confidence pseudo-label regions; The similarity between the anchor point and the positive and negative sample features is obtained based on the anchor point comparison pair, and the pixel-level contrast loss is calculated; the pixel-level contrast loss includes: supervision loss, consistency loss and contrast loss.

5. A crack image segmentation system for large building surfaces, characterized in that, include: The acquisition module is used to acquire the original image of the building surface of the target area to be detected; Annotation is performed on the original image of the building surface to obtain an annotated image and an unannotated image of the building surface; The heterogeneous dual-branch network construction module is used to extract multi-dimensional feature vectors from the labeled and unlabeled images of the building surface through a heterogeneous network composed of a convolutional neural network and a Transformer decoder branch, and generate image prediction results. The multidimensional feature vector includes: feature maps of labeled data and feature maps of unlabeled images; the image prediction result includes: crack segmentation probability feature map and comparison result; wherein... The annotation image of the building surface is processed by encoding and decoding and feature extraction to obtain a feature map with annotation data; specifically, it includes: extracting multi-scale features of the annotation image of the building surface; inputting the image features extracted by the encoder into the CNN decoder and outputting a pixel-level crack segmentation probability map corresponding to the annotation image of the building surface; updating parameters based on the pixel-level crack segmentation probability map and mapping the multi-scale features of the annotation image to generate a feature map with annotation data. A multi-scale perturbation strategy is applied to the unlabeled image for prediction, resulting in a multi-scale perturbation prediction result. This strategy includes strong perturbation paths and weak perturbation paths. Specifically, it involves: using a convolutional neural network to extract encoder features and a CNN decoder to reconstruct the unlabeled image, obtaining a crack segmentation probability map and generating a crack segmentation probability feature map; using a Transformer model to extract and decode features from the feature map of the unlabeled image, obtaining a crack segmentation prediction map, which serves as a pseudo-label for the unlabeled image; comparing the pseudo-label with the crack segmentation probability feature map to constrain the model's training on the unlabeled image, obtaining a comparison result; and using the crack segmentation probability feature map and the comparison result as the multi-scale perturbation prediction result. The optimization supervision module is used to dynamically adjust the prediction based on the image prediction results and to constrain them using a consistency supervision mechanism to obtain optimized prediction labels. This includes: integrating data based on the crack segmentation probability feature map to generate an optimized pseudo-label mask; sampling unlabeled pixels based on the optimized pseudo-label mask to obtain a set of unlabeled data pixels; extracting high-confidence predicted pixels from the set of unlabeled data pixels, and using these high-confidence predicted pixels as the optimized prediction labels. The pixel-level comparison module is used to perform pixel-level comparison learning between the multidimensional feature vector and the optimized predicted label, so as to achieve feature alignment between the labeled data feature map and the unlabeled image and obtain the final crack segmentation result. The output and inference module is used to perform inference based on the final crack segmentation result to obtain a crack segmentation probability map, so as to realize crack detection and evaluation on the building surface.

6. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing a computer program stored in the memory to cause the electronic device to perform the crack image segmentation method for large building surfaces as described in any one of claims 1 to 4.

Images