Large building surface crack image segmentation method, system and equipment

Abstract

The invention provides a crack image segmentation method, system and device for a large building surface. The method comprises the following steps: acquiring a building surface original image of a to-be-detected target area; extracting a multi-dimensional feature vector of the original building surface image through a heterogeneous network formed by a convolutional neural network and a Transform decoder branch, and generating an image prediction result; carrying out dynamic adjustment based on an image prediction result, and carrying out constraint by adopting a consistency supervision mechanism to obtain an optimized prediction label; pixel-level contrast learning is carried out on the multi-dimensional feature vector and the optimized prediction label, and a final crack segmentation result is obtained; and reasoning based on the final crack segmentation result to obtain a crack segmentation probability graph. According to the method, automatic identification and accurate segmentation are carried out on the surface cracks of key infrastructures such as a dam through an artificial intelligence algorithm, the crack detection precision is improved under the condition of limited labeled data, the model robustness is enhanced, the facility operation and maintenance efficiency is optimized, and the structure safety is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image data processing, and relates to a building extraction method, in particular to a large building surface crack image segmentation method, system and device. BACKGROUND

[0002] As a key water conservancy infrastructure of the country, the dam plays a key role in the fields of water resources regulation, flood control and disaster resistance, energy supply and shipping development. With the increase of the running time of a large number of dams, the performance degradation problem caused by the aging of structural materials and long-term load effect gradually appears. The continuous and effective safety monitoring and operation and maintenance management of the dam have become the focus of the industry. Among them, the early detection and accurate evaluation of the surface crack of the dam, which is a common structural defect, is an important technical link for implementing the whole life cycle health management of the dam. The continuous expansion of the crack may affect the integrity of the dam body and aggravate the internal seepage risk, which poses a potential threat to the long-term safe operation of the dam.

[0003] At present, a large number of large and medium-sized dams have been built in China, and the running time of some of the projects has approached or exceeded the design benchmark period. In such projects, if structural defects cannot be identified and disposed of in time, the safety state of the dam may be affected, and the social and economic activities and the ecological environment of the downstream area may be adversely affected.

[0004] Among the many hidden dangers affecting the safety of the dam structure, the early identification and dynamic evaluation of the surface structural crack are particularly critical. The formation mechanism of the crack is complex, which may be limited by material aging and temperature stress, or may be caused by geological subsidence, water pressure impact and sudden natural disasters. Once the crack develops out of control, not only the overall stiffness of the structure is weakened, but also it may become a seepage channel and aggravate the internal force imbalance of the dam body. Therefore, improving the automation and precision level of crack detection is the technical basis for realizing the whole cycle of dam "prediction, early warning, prediction and control" intelligent supervision.

[0005] At present, for the detection of dam cracks, the traditional manual visual inspection and local instrument assisted detection methods are mainly relied on in engineering practice. For example, detection personnel will use laser scanners, infrared thermal imagers or ultrasonic range finders and other equipment to conduct fixed-point or sampling inspection on the surface of the dam. However, these existing technologies have the following significant defects: (1) Low efficiency and high cost: manual inspection method needs professional personnel to go to the scene, which has high labor intensity and long detection period, and for the large dam structure, it is difficult to realize comprehensive and high-frequency coverage, resulting in high labor cost and time cost; (2) Real-time performance is poor and subjective: Artificial detection and instrument-assisted detection are mostly discontinuous and periodic, which cannot monitor the dynamic development of cracks in real time. At the same time, the detection results are heavily dependent on the experience and subjective judgment of the detection personnel, which is prone to missed detection and misjudgment, and it is difficult to ensure the consistency and objectivity of the data; (3) Weak environmental adaptability and insufficient robustness: The surface environment of the dam is complex and changeable, and there are often uneven illumination, shadow blocking, surface stains, low-contrast background and other interference factors. The recognition accuracy and stability of the traditional detection methods (including some algorithms based on simple image processing) will decrease dramatically when dealing with these complex scenes, which is difficult to meet the all-weather and high-reliability monitoring requirements.

[0006] In recent years, with the rapid development of artificial intelligence and computer vision technology, image recognition technology based on deep learning provides a new solution for the automatic detection of dam cracks. In particular, deep learning models represented by CNN (Convolutional Neural Network), Vision Transformer, etc. have shown superior performance in crack segmentation and positioning tasks due to their powerful feature extraction and pattern recognition capabilities. However, when applying such advanced models to practical engineering, there is still a core bottleneck: Existing mainstream deep learning models highly depend on large-scale and high-quality pixel-level labeled data for supervised training. In the specific application scenario of dam crack detection, it is extremely difficult to obtain such labeled data. On the one hand, professional labeling requires a lot of manpower and time, and the labeling cost is extremely high; on the other hand, due to the complex structure of the dam, the diverse and uneven distribution of cracks, it is almost impossible to obtain a complete dataset covering various working conditions and crack types. This "data scarcity" problem greatly limits the performance and deployment of existing advanced models in practical engineering, resulting in poor generalization ability and insufficient robustness, which makes it difficult to meet the urgent needs of intelligent water conservancy engineering for automation, precision and lightweight deployment.

[0007] In summary, in the prior art, traditional detection methods are low in efficiency, high in cost and poor in adaptability; while the automatic detection method based on deep learning has superior performance, but is limited by the strong dependence on massive labeled data. Therefore, how to provide an intelligent algorithm framework for high-precision and high-robustness dam crack segmentation under the condition of extremely scarce labeled data, to overcome the above-mentioned defects of the prior art, has become a technical problem to be solved in the field. SUMMARY

[0008] The purpose of the present application is to provide a large building surface crack image segmentation method, system and device, which is used to solve the problem of high precision and high efficiency automatic identification and segmentation of dam surface cracks under the condition of extremely scarce labeled data in the prior art, and to overcome the problems of strong artificial dependence, high labeling cost and poor environmental adaptability of traditional detection methods and existing deep learning models.

[0009] In a first aspect, the present application provides a large building surface crack image segmentation method, comprising the following steps: obtaining a building surface original image of a target area to be detected; extracting a multi-dimensional feature vector of the building surface original image through a heterogeneous network composed of a convolutional neural network and a Transformer decoder branch, and generating an image prediction result; the multi-dimensional feature vector includes a labeled data feature map and an unlabeled image feature map; the image prediction result includes a crack segmentation probability feature map and a comparison result; based on the image prediction result, dynamic adjustment is performed and a consistency supervision mechanism is used for constraint to obtain an optimized prediction label; pixel-level comparison learning is performed on the multi-dimensional feature vector and the optimized prediction label to realize alignment of the labeled data feature map and the unlabeled image feature map, and a final crack segmentation result is obtained; based on the final crack segmentation result, inference is performed to obtain a crack segmentation probability map, so as to realize building surface crack detection and evaluation.

[0010] In an implementation form of the first aspect, the multi-dimensional feature vector of the building surface original image is extracted through the heterogeneous network composed of the convolutional neural network and the Transformer decoder branch, including: based on the building surface original image, a building surface labeled image and an unlabeled image are obtained; based on the building surface labeled image, encoding and decoding processing and feature extraction are performed to obtain a labeled data feature map; a multi-scale perturbation strategy is used to predict the unlabeled image to obtain a multi-scale perturbation prediction result.

[0011] In the implementation form, the joint modeling of crack details and global structure is realized by fusing CNN and Transformer two different decoder architectures, and the multi-scale perception ability and adaptability to complex texture of the model are enhanced.

[0012] In an implementation form of the first aspect, the obtaining the feature map of the labeled data based on the coding processing and the feature extraction on the building surface labeled image comprises: performing feature extraction on the building surface labeled image to obtain labeled image multi-scale features; inputting the image features extracted by the encoder into a CNN decoder, and outputting a pixel-level crack segmentation probability map corresponding to the building surface labeled image by the CNN decoder; performing parameter updating based on the pixel-level crack segmentation probability map, and mapping the labeled image multi-scale features to generate the feature map of the labeled data.

[0013] In an implementation form of the first aspect, the multi-scale perturbation strategy comprises a strong perturbation path and a weak perturbation path; the obtaining the multi-scale perturbation prediction result by predicting the unlabeled image based on the multi-scale perturbation strategy comprises: performing encoder feature extraction and CNN decoder reconstruction processing on the unlabeled image by a convolutional neural network to obtain a crack segmentation probability map of the unlabeled image and generate a crack segmentation probability feature map; performing feature extraction and decoding on the feature map of the unlabeled image by a Transformer model to obtain a crack segmentation prediction map as a pseudo label of the unlabeled image; comparing the pseudo label with the crack segmentation probability feature map to constrain the training of the model on the unlabeled image to obtain a comparison result; and taking the crack segmentation probability feature map and the comparison result as the multi-scale perturbation prediction result.

[0014] In an implementation form of the first aspect, the obtaining the optimized prediction label based on the dynamic adjustment of the image prediction result and the constraint by the consistency supervision mechanism comprises: generating an optimized pseudo label mask map based on the crack segmentation probability feature map; performing unlabeled pixel sampling based on the optimized pseudo label mask map to obtain a pixel sample set of unlabeled data; extracting high-confidence predicted pixels from the pixel sample set of the unlabeled data, and taking the high-confidence predicted pixels as the optimized prediction label.

[0015] In the implementation form, the pseudo label optimization mechanism based on uncertainty estimation is used to solve the problem of pseudo label noise interference and improve the utilization efficiency of unlabeled data in semi-supervised learning. Furthermore, the consistency supervision mechanism is used to improve the robustness of the model to input perturbation, ensure that the prediction results of the two branches under different views remain consistent, and thus enhance the semantic alignment ability of the model.

[0016] In an implementation form of the first aspect, the generating the optimized pseudo-label mask based on the crack segmentation probability feature map comprises: obtaining a fused average probability map based on the crack segmentation probability feature map; calculating a predicted entropy value of each pixel in the fused average probability map; and performing dynamic threshold screening according to the predicted entropy value of each pixel to obtain the optimized pseudo-label mask.

[0017] In an implementation form of the first aspect, the sampling unlabeled pixels based on the optimized pseudo-label mask to obtain a pixel sample set of unlabeled data comprises: extracting a plurality of samples from the optimized pseudo-label mask to construct the pixel sample set of unlabeled data; the pixel sample set of unlabeled data comprises positive samples and negative samples; the positive samples are pixel features in a crack region indicated by the mask, and the negative samples are pixel features sampled from a background region outside the mask.

[0018] In an implementation form of the first aspect, the pixel-level contrast learning of the multi-dimensional feature vector and the optimized predicted label to realize feature alignment of the feature map of the labeled data and the feature of the unlabeled image to obtain a final crack segmentation result comprises: extracting a high-confidence pseudo-label region according to the multi-dimensional feature vector and the optimized predicted label; constructing an anchor point contrast pair based on the high-confidence pseudo-label region; obtaining a similarity between an anchor point and a positive sample feature and a negative sample feature according to the anchor point contrast pair, and calculating a pixel-level contrast loss; the pixel-level contrast loss comprises a supervision loss, a consistency loss and a contrast loss.

[0019] In the implementation form, the pixel-level contrast learning module is used to improve the intra-class cohesion and inter-class separation of the model in the feature space, and enhance the feature distinguishing ability between the crack and the background.

[0020] In a second aspect, the present application provides a large building surface crack image segmentation system, comprising: an acquisition module configured to acquire a building surface original image of a target area to be detected; a heterogeneous dual-branch network construction module configured to extract a multi-dimensional feature vector of the building surface original image by a heterogeneous network composed of a convolutional neural network and a Transformer decoder branch, and generate an image prediction result; the multi-dimensional feature vector comprises a labeled data feature map and an unlabeled image feature map; the image prediction result comprises a crack segmentation probability feature map and a comparison result; an optimization supervision module configured to dynamically adjust based on the image prediction result and constrain by a consistency supervision mechanism to obtain an optimized prediction label; a pixel-level comparison module configured to perform pixel-level comparison learning on the multi-dimensional feature vector and the optimized prediction label to realize alignment of the labeled data feature map and the unlabeled image feature map, and obtain a final crack segmentation result; and an output and reasoning module configured to perform reasoning based on the final crack segmentation result, acquire a crack segmentation probability map, and realize crack detection and evaluation of the building surface.

[0021] In a last aspect, the present application provides an electronic device, comprising: a processor and a memory. The memory is configured to store a computer program; the processor is connected with the memory and is configured to execute the computer program stored in the memory, so that the large building surface crack image segmentation device executes the large building surface crack image segmentation method.

[0022] As described above, the large building surface crack image segmentation method, system and device of the present application have the following beneficial effects: (1) The large building surface crack image segmentation method provided by the present application constructs a dual-branch heterogeneous network architecture composed of a convolutional neural network and a Transformer decoder, which can process strong disturbance views and weak disturbance views of the same unlabeled image respectively, can fully exploit the multi-scale and multi-form features of cracks in the image, and improves the recognition ability of the model for small cracks and irregular edges.

[0023] (2) The dynamic pseudo-label screening and consistency supervision mechanism in the present application, after generating a pseudo-label by the Transformer branch, estimates the pixel-level confidence by using the entropy value of the prediction result, thereby eliminating uncertain areas and only retaining high-confidence areas for training, effectively suppressing pseudo-label noise. At the same time, by using a consistency loss function between the outputs of the dual branches, the semantic consistency learning of the model under different disturbance conditions is strengthened, and the stability of the model is improved.

[0024] (3) The pixel-level contrast learning module based on uncertainty guidance introduced in the present application promotes the aggregation of similar crack pixels and the separation of dissimilar pixels in the feature space, thereby improving the model's ability to distinguish between cracks and background boundaries.

[0025] (4) The training target function design method of joint multi-source supervision in the present application comprehensively considers three types of loss functions: label supervision, consistency learning, and contrast learning, and jointly optimizes the model from three dimensions: pixel accuracy, semantic stability, and feature discriminability. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 A schematic diagram of a hardware application scenario in an embodiment of the crack image segmentation method for large building surfaces described in the present application.

[0027] Figure 2 A flowchart of an embodiment of the crack image segmentation method for large building surfaces described in the present application.

[0028] Figure 3 A schematic diagram of the data processing flow in the crack image segmentation method for large building surfaces described in the present application.

[0029] Figure 4 A flowchart of S2 in the crack image segmentation method for large building surfaces described in the present application.

[0030] Figure 5 A flowchart of S22 in the crack image segmentation method for large building surfaces described in the present application.

[0031] Figure 6 A flowchart of S23 in the crack image segmentation method for large building surfaces described in the present application.

[0032] Figure 7 A flowchart of S3 in the crack image segmentation method for large building surfaces described in the present application.

[0033] Figure 8 A flowchart of S31 in the crack image segmentation method for large building surfaces described in the present application.

[0034] Figure 9 A flowchart of S4 in the crack image segmentation method for large building surfaces described in the present application.

[0035] Figure 10 A CPCL structure diagram in an embodiment of the crack image segmentation method for large building surfaces described in the present application.

[0036] Figure 11The diagram shows experimental results of an embodiment of the method for segmenting crack images on the surface of a large building as described in this application.

[0037] Figure 12 The diagram shown is a schematic representation of the principle structure of the large building surface crack image segmentation system described in this application in one embodiment.

[0038] Figure 13 The diagram shown is a schematic representation of the electronic device described in this application.

[0039] Component designation explanation 11 Data acquisition module 12 Data processing module 13 Dual-branch heterogeneous module 1301 Dual-branch heterogeneous network structure module 1302 Multi-scale perturbation design module 14 Consistency learning module 15 Contrastive learning module 121 Acquisition module 122 Heterogeneous dual-branch network construction module 123 Optimization supervision module 124 Pixel-level contrast module 125 Output and inference module 131 Processor 132 Memory Detailed Implementation

[0040] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0041] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0042] The following embodiments of this application provide a method for segmenting crack images on the surface of large buildings, which solves the problem of high-precision and high-efficiency automatic identification and segmentation of cracks on the surface of dams under the condition of extremely scarce labeled data in the prior art. This overcomes the problems of traditional detection methods and existing deep learning models, such as strong dependence on manual labor, high labeling costs, and poor environmental adaptability.

[0043] like Figure 1 As shown, the hardware application scenario of the crack image segmentation method for large building surfaces described in this application is illustrated in one embodiment. Specifically, it includes: a data acquisition module 11, a data processing module 12, a dual-branch heterogeneous module 13, a consistency learning module 14, and a contrastive learning module 15.

[0044] The data acquisition module 11 is configured to acquire a building surface original image of a target area to be detected. The data processing module 12 is configured to process the building surface original image of the target area to be detected and extract key features from the image. The dual-branch heterogeneous module 13 includes a dual-branch heterogeneous network structure module 1301 and a multi-scale perturbation design module 1302, which are configured to construct a dual-branch architecture composed of a convolutional neural network (CNN) and a Transformer decoder, and process strong perturbation views and weak perturbation views of the same unlabeled image, respectively. Through differential data enhancement strategies (such as random cropping, color perturbation, CutMix, etc.), the changes of the image in the real environment are simulated, and multi-view feature expression is realized. The two branches share the encoder but use heterogeneous decoders, thereby improving the robustness and feature extraction capability of the model while maintaining the stability of the structural semantics. The consistency learning module 14 is configured to use the pseudo-label generated by the Transformer branch, combine the entropy value of the prediction distribution, and dynamically select high-confidence regions as pseudo-supervision signals. In the early stage of training, a strict confidence threshold is used to control noise interference, and in the later stage, the threshold is gradually relaxed to expand the supervision range. At the same time, a cross-branch consistency loss function is designed to make the outputs of the CNN branch and the Transformer branch under different perturbation conditions remain semantically consistent, thereby enhancing the stability of the model in representing crack structures. The contrast learning module 15 is configured to introduce a contrast learning module in the CNN branch, sample positive and negative samples for pixel-level embedding of high-confidence pseudo-label regions, construct an InfoNCE-based contrast loss function, and promote the aggregation of similar pixels and the separation of dissimilar pixels in the feature space, thereby improving the discrimination between cracks and backgrounds. To improve computational efficiency, a dynamically updated feature memory queue is introduced to store prototype embedding vectors, which maintains the stability of training while reducing positive and negative sample selection errors.

[0045] In summary, the present application realizes high-precision crack detection under the condition of limited labeled samples, reduces the cost of manual labeling, has strong robustness and good boundary detail perception ability, and is suitable for complex textures and low-contrast images. It can be deployed in inspection systems for large structures such as dams, bridges and tunnels, and helps intelligent management of infrastructure.

[0046] The large building surface crack image segmentation method provided in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0047] Please refer to Figure 2 and Figure 3 , respectively showing the flowchart of the large building surface crack image segmentation method in an embodiment and the data processing flowchart of the large building surface crack image segmentation method. As Figure 2 and Figure 3As shown, the embodiment provides a large building surface crack image segmentation method.

[0048] The large building surface crack image segmentation method can achieve high-precision crack identification under the condition of limited labeled samples, reduce the cost of manual labeling, has strong robustness and good boundary detail perception ability, and is suitable for complex texture and low-contrast images. It can be deployed in large structure inspection systems such as dams, bridges, and tunnels to help intelligent management of infrastructure.

[0049] The large building surface crack image segmentation method specifically includes the following steps: S1, obtaining the original image of the building surface of the target area to be detected.

[0050] In this embodiment, the building facade in the target area to be detected is divided into several manageable areas (such as by floor, by facade), and each area is numbered to ensure no omission, and fine building area division and planning is performed. Then, select appropriate equipment according to the planning and perform standardized image acquisition. Then, the massive images collected are systematically organized and labeled.

[0051] Specifically, first, based on the detection target, the building facade to be detected is systematically divided into several manageable grids or areas (such as by floor, orientation, or structural unit), and each area is given a unique identifier. This step aims to ensure that subsequent collection work is complete and covers all areas, while developing detailed safety plans and collection paths.

[0052] Then, select appropriate equipment according to the planning and perform standardized image acquisition. For building height and accessibility, select devices such as drones or high-definition cameras, and under uniform lighting conditions, shoot along the predetermined path with the lens perpendicular to the building surface. Ensure high overlap between adjacent images when collecting, and take multiple scale close-up shots of suspected defect areas with scale included to obtain high-definition, distortion-free original images.

[0053] That is, the target area and shooting angle can be determined according to the detection requirements, and then high-resolution cameras, drones, mobile phones, and other image collection devices can be used to take multi-angle, full-coverage shots of the building surface under suitable lighting conditions (such as no direct sunlight or uniform artificial light during the day), ensuring clear, unobstructed, and no significant distortion. The collected images are stored in raw format (such as RAW or high-resolution JPEG) for subsequent image processing, defect identification, or three-dimensional modeling analysis.

[0054] S2, extracts a multi-dimensional feature vector of the building surface original image through a heterogeneous network composed of a convolutional neural network and a Transformer decoder branch, and generates an image prediction result. The multi-dimensional feature vector includes a feature map of labeled data and a feature map of unlabeled images; and the image prediction result includes a crack segmentation probability feature map and a comparison result.

[0055] In this embodiment, a double-branch heterogeneous network structure and a multi-scale perturbation design are adopted. That is, a double-branch architecture composed of a convolutional neural network and a Transformer decoder is constructed to process strong perturbation views and weak perturbation views of the same unlabeled image respectively. Through differentiated data enhancement strategies (such as random cropping, color perturbation, CutMix, etc.), the changes of the image in the real environment are simulated to realize multi-view feature expression. The two branches share the encoder but use heterogeneous decoders, thereby improving the robustness and feature extraction capability of the model while maintaining the stability of the structural semantics.

[0056] Referring to Figure 4 , a flowchart of S2 in the crack image segmentation method for large building surfaces described in the present application is shown. As Figure 4 indicated, the S2 includes the following steps: S21, based on the building surface original image, obtaining a building surface labeled image and an unlabeled image.

[0057] In this embodiment, the collected building surface original images are systematically data-arranged and labeled. After backing up the image data, the images are classified according to the divided areas, and the files are renamed using standardized rules. In the data labeling process, key information labels such as region number, shooting time, orientation, etc. are added to each image, thereby forming a set of original image data with clear structure and complete information, laying a solid foundation for subsequent detection and analysis.

[0058] S22, based on the building surface labeled image, performing encoding and decoding processing and feature extraction to obtain a feature map of labeled data. Referring to Figure 5 , a flowchart of S22 in the crack image segmentation method for large building surfaces described in the present application is shown. As Figure 5 indicated, the S22 includes the following steps: S221, based on the building surface labeled image, performing feature extraction to obtain a multi-scale feature of the labeled image; S222, inputting the image features extracted by the encoder into a CNN decoder, and outputting a pixel-level crack segmentation probability map corresponding to the building surface labeled image by the CNN decoder; S223, updating parameters based on the pixel-level crack segmentation probability map, and mapping the multi-scale features of the labeled image to generate a feature map of labeled data.

[0059] In this embodiment, the image semantics based on the heterogeneous double-branch structure are modeled. The backbone encoder adopts a shared CNN structure, which is connected to a convolutional decoder and a Transformer decoder, respectively. The Transformer decoder processes weakly perturbed images, maintaining structural integrity; the CNN decoder processes strongly perturbed images, enhancing detail robustness.

[0060] Specifically, the training image with real crack annotation is input into the encoder network of the CSCL (Computer Supported Collaborative Learning) model to extract multi-scale features. Then the encoder features are sent to the CNN decoder to output the corresponding crack segmentation probability map. The supervised loss is calculated based on the probability map and the real annotation of the image, which is used to update the model parameters supervisedly. Meanwhile, the projection head connected after the encoder is used to map the extracted features to generate a feature map of labeled data, providing a basis for subsequent pixel-level contrastive learning.

[0061] S23, using a multi-scale perturbation strategy to predict the unlabeled image to obtain a multi-scale perturbation prediction result. The multi-scale perturbation strategy includes a strong perturbation path and a weak perturbation path. Please refer to Figure 6 , which shows the flowchart of S23 in the crack image segmentation method of the large building surface described in this application. As Figure 6 shown, the S23 includes the following steps: S231, performing encoder feature extraction and CNN decoder reconstruction processing on the unlabeled image through a convolutional neural network to obtain a crack segmentation probability map of the unlabeled image and generate a crack segmentation probability feature map; S232, extracting and decoding the feature map features of the unlabeled image through a Transformer model to obtain a crack segmentation prediction map as a pseudo label of the unlabeled image; S233, comparing the pseudo label with the crack segmentation probability feature map to constrain the training of the model on the unlabeled image to obtain a contrast result; S234, taking the crack segmentation probability feature map and the contrast result as a multi-scale perturbation prediction result.

[0062] In this embodiment, two views of the input image are generated: weakly perturbed images for pseudo-label generation, and strongly perturbed images for feature diversity learning. Weak perturbations include random scaling, cropping, flipping, etc., and strong perturbations include color jittering, Gaussian blur, CutMix, etc.

[0063] Specifically, the unlabeled image is simultaneously input into two parallel processing paths of strong perturbation and weak perturbation. Among them, the strong perturbation path is a CNN branch (the structure is the same as the labeled data branch in step S2), which passes through the encoder and the CNN decoder to obtain the crack segmentation probability map of the unlabeled image, and generates the corresponding feature map through the projection head; the weak perturbation path is a Transformer branch, which inputs the same unlabeled image after applying weak data augmentation into the Transformer model to extract features and generate a crack segmentation prediction map as the pseudo-label of the unlabeled image.

[0064] Then, the pseudo-label generated by the weak perturbation path is compared with the segmentation probability map output by the strong perturbation path, and a consistency loss is calculated to constrain the training of the model on the unlabeled data. At the same time, the output results of the strong perturbation and weak perturbation paths above provide a basis for the next step of pseudo-label fusion.

[0065] It should be noted that the present application constructs a dual-branch heterogeneous network architecture composed of a convolutional neural network (CNN) and a Transformer decoder, which can process strong perturbation views and weak perturbation views of the same unlabeled image. The CNN branch focuses on processing local detail information of the image after image enhancement (such as color perturbation, Gaussian blur, etc.), while the Transformer branch maintains the global semantic information of the image structure (such as flipping, cropping, etc. Light perturbation), both complement each other, fully excavate the multi-scale and multi-morphology features of the cracks in the image, and improve the recognition ability of the model for small cracks and irregular edges.

[0066] S3, based on the image prediction result, dynamic adjustment is performed, and a consistency supervision mechanism is used for constraint to obtain an optimized prediction label. Please refer to Figure 7 , which shows the flowchart of S3 in the crack image segmentation method of the large building surface according to the present application. As Figure 7 indicated, the S3 comprises the following steps: S31, based on the crack segmentation probability feature map, data integration is performed to generate an optimized pseudo-label mask map. Please refer to Figure 8 , which shows the flowchart of S31 in the crack image segmentation method of the large building surface according to the present application. As Figure 8 indicated, the S31 comprises the following steps: S311, based on the crack segmentation probability feature map, a fused average probability map is obtained; S312, calculating a prediction entropy value of each pixel in the fused average probability map; S313, performing dynamic threshold screening according to the prediction entropy value of each pixel to obtain an optimized pseudo-label mask.

[0067] In this embodiment, the prediction results of the fusion of strong disturbance and weak disturbance paths are preferably used to generate an optimized pseudo-label.

[0068] The weak disturbance image is predicted using the Transformer branch to output a class probability map. The prediction entropy value of each pixel is calculated as an uncertainty indicator, and the lower the entropy value, the more reliable the prediction. At the same time, an initial confidence threshold is set, and only the pixels with an entropy value lower than the threshold are retained as effective pseudo-labels to participate in training. As the training progresses, the threshold is gradually reduced, allowing the model to use more high-confidence pseudo-label regions to expand the coverage of pseudo-supervision.

[0069] Specifically, the average probability map is obtained by averaging the crack segmentation probability maps output by the strong disturbance CNN branch and the weak disturbance Transformer branch in the preceding steps. The entropy value distribution of the average probability map is calculated to measure the uncertainty of the prediction of each pixel. According to a preset entropy threshold, low-entropy (high-confidence) pixel regions are selected to generate an optimized pseudo-label mask, retaining only the crack pixels predicted with high confidence by the model. The optimized pseudo-label mask improves the accuracy of the pseudo-labels of the unlabeled data and provides a basis for subsequent pixel sample sampling.

[0070] S32, based on the optimized pseudo-label mask, sampling unlabeled pixels to obtain a pixel sample set of unlabeled data; S33, extracting high-confidence predicted pixels from the pixel sample set of unlabeled data, and using the high-confidence predicted pixels as optimized prediction labels.

[0071] In this embodiment, a number of samples are extracted from the optimized pseudo-label mask to construct a pixel sample set of unlabeled data. The pixel sample set of unlabeled data includes positive samples and negative samples. The positive samples are pixel features in the crack region indicated by the mask, and the negative samples are pixel features sampled from the background region outside the mask.

[0072] According to the optimized pseudo-label mask obtained in the preceding steps, high-confidence pixel samples are selected from the feature map of the unlabeled data for contrastive learning.

[0073] The application uses cross-entropy loss to constrain the outputs of the CNN branch and the Transformer branch on the same unlabeled sample. In the case where only part of the pseudo-labels are available, consistency supervision provides stable and low-bias training signals for the model.

[0074] Specifically, a certain number of pixel features in the crack area indicated by the mask are randomly sampled as positive samples, and pixel features in the background area outside the mask are sampled as negative samples to form a pixel sample set of unlabeled data. By collecting only the pixels predicted by the model with high confidence, it is ensured that the contrast learning stage focuses on reliable pseudo-label samples. The above-selected pixel samples will be used in the next pixel-level contrast learning process.

[0075] It should be noted that a dynamic pseudo-label screening and consistency supervision mechanism is adopted in the present application. After generating the pseudo-labels by the Transformer branch, the pixel-level confidence is estimated using the entropy value of the prediction results, thereby eliminating uncertain areas and only retaining high-confidence areas for training, effectively suppressing pseudo-label noise. As the training progresses, the confidence threshold is gradually reduced, allowing more pseudo-label areas to be included in the supervision range, thereby realizing the transition of the model from "robust learning" to "extensive generalization". At the same time, the consistency loss function between the outputs of the two branches is used to strengthen the semantic consistency learning of the model under different perturbation conditions, thereby improving the stability of the model.

[0076] S4, performing pixel-level contrast learning on the multi-dimensional feature vector and the optimized prediction label to align the feature maps of the labeled data and the unlabeled image, and obtaining a final crack segmentation result. Please refer to Figure 9 , which shows the flowchart of S4 in the crack image segmentation method for large building surfaces according to the present application. As shown in Figure 9 , the S4 comprises the following steps: S41, extracting a high-confidence pseudo-label area according to the multi-dimensional feature vector and the optimized prediction label; S42, constructing an anchor point contrast pair based on the high-confidence pseudo-label area; S43, obtaining the similarity between the anchor point and the positive sample feature and the negative sample feature according to the anchor point contrast pair, and calculating a pixel-level contrast loss; the pixel-level contrast loss comprises a supervision loss, a consistency loss, and a contrast loss.

[0077] Please refer to Figure 10 , which shows the CPCL structure diagram in an embodiment of the crack image segmentation method for large building surfaces according to the present application.

[0078] In this embodiment, the feature map of the labeled data, the feature map of the unlabeled image, and the pixel samples and their pseudo-labels obtained by sampling are input into a pixel-level contrast learning (CPCL) module. Among them, the pixel features with true labels in the labeled data feature map are taken as anchor points, the unlabeled pixel features of the corresponding class (crack or background) are regarded as positive samples, and the unlabeled pixel features not belonging to the class are regarded as negative samples.

[0079] The CPCL module measures the similarity between the anchor point feature and the positive and negative sample features, and calculates the pixel-level contrast loss: the similarity between the anchor point and the positive sample feature is improved, and the similarity between the anchor point and the negative sample feature is reduced. Under this contrast learning constraint, the feature alignment of labeled data and unlabeled data is realized, and the discrimination ability of the model for cracks and background pixels is enhanced. The above contrast loss, together with the supervised loss and consistency loss, further optimizes the model parameters, laying a foundation for the final high-precision crack segmentation output.

[0080] Specifically, embedding feature extraction and class labeling are performed. The low-dimensional feature representation of each pixel point is extracted in the CNN branch and corresponds to the pseudo-label class. Only the pixel embedding of the high-confidence pseudo-label region is used for contrast learning to avoid noise interference.

[0081] Then, a positive and negative sample pair is constructed. Pixels of the same class are used as positive samples, and pixels of different classes are used as negative samples to construct an anchor point contrast pair. The InfoNCE loss function is used to calculate the contrast loss of each anchor point to guide the model to learn discriminative embedding representation.

[0082] Next, the feature memory queue is maintained. A sliding update prototype queue is used to store historical embedding vectors to alleviate the batch-to-batch sample offset problem. The feature queue improves sample diversity and training stability while reducing memory and computing costs.

[0083] The joint loss function optimization strategy integrates multiple supervision signals into a unified optimization target, effectively coordinating the training weights among label supervision, consistency learning, and contrast learning.

[0084] The total loss function consists of a supervised loss, a consistency loss, and a contrast loss.

[0085] The calculation formula of the pixel-level contrast loss is:

[0086] wherein, Lsup represents the supervised loss value; Lcon represents the consistency loss value; Lcon represents the contrast loss function value; , Lsup, Lcon, and Lcon all represent loss weight parameters that are dynamically adjusted according to the training stage, with a focus on consistency in the early stage and an emphasis on contrast learning in the later stage.

[0087] That is, the present application considers three types of loss functions, namely label supervision, consistency learning, and contrast learning, and jointly optimizes the model from three dimensions of pixel accuracy, semantic stability, and feature discriminability.

[0088] It should be noted that this application introduces a pixel-level contrastive learning module based on uncertainty guidance. By selecting high-confidence regions in the pseudo-labels as anchor points, positive and negative sample pairs are constructed, and InfoNCE loss is used for feature embedding optimization. This module promotes the clustering of similar crack pixels in the feature space and the separation of dissimilar pixels, thereby improving the model's ability to distinguish cracks from background boundaries. To reduce computational redundancy during training, this invention further designs a prototype feature memory queue mechanism to dynamically maintain the embedding vectors of high-quality samples, achieving a dual optimization of training efficiency and feature generalization ability.

[0089] S5. Based on the final crack segmentation result, reasoning is performed to obtain a crack segmentation probability map in order to realize crack detection and evaluation on the building surface.

[0090] In this embodiment, after the above training process, the model of the present invention finally uses the output of the CNN decoder as the final result of crack segmentation. During the inference phase, the image to be detected is input into the trained encoder and CNN decoder to directly obtain the crack segmentation probability map. The segmentation result output by the CNN decoder is used for actual crack detection and evaluation, thus completing the entire CSCL crack segmentation process.

[0091] Specifically, the inference process simplifies the model structure to improve deployment efficiency, facilitating real-time monitoring tasks on edge devices or engineering systems. Only the CNN branch is retained to generate the final crack segmentation results, reducing model complexity and inference latency. The CNN branch, as the main output path, offers advantages such as low computational overhead and simple deployment, making it suitable for real-time detection scenarios in engineering sites.

[0092] It should be noted that this strategy can achieve rapid and low-resource-consumption engineering deployment without sacrificing accuracy.

[0093] The following example uses a set of experimental data to illustrate this.

[0094] Please see Figure 11 The image shown is a schematic diagram of the experimental results of the method for segmenting crack images on the surface of a large building as described in this application in one embodiment.

[0095] The model performance verification here uses a real crack image dataset to comprehensively evaluate the method of this invention, verifying its stability and superiority under different amounts of labeled data. The main contents include the following: (1) Dataset partitioning and annotation settings Based on the CRACK500 dataset, three semi-supervised scenarios with annotation ratios of 5%, 12.5%, and 25% are constructed. The performance of the supervised learning model and mainstream semi-supervised contrast methods such as MT (Mean Teacher), UAMT (Uncertainty-aware Mean Teacher), CPS (Cross Pseudo Supervision), UGPCL (Uncertainty-Guided Pixel-wise Contrastive Learning), CSCL (Crack Segmentation on via Contrastive Learning), and TCSM_V2 (Transformation-consistent self-ensembling) is compared.

[0096] (2) Comparison of experimental results As shown in Figure 11 , the qualitative visualization comparison on the CRACK500 dataset is as follows: from left to right, the input image, the labeled picture (Ground Truth), and the segmentation results of the five methods (CSCL, UAMT, CPS, MT, and TCSM_V2) are given. As can be seen from Figure 11 , each of the above methods can generally find the crack region, but there are differences in performance, such as: MT and TCSM_V2 are prone to "breakage / incoherence" on thin and low-contrast cracks; the CPS mask is smoother, but occasionally misjudges background textures as cracks; UAMT has a tendency to over-segment weak structures; compared with the above, the boundaries of CSCL are clearer and the connectivity is better.

[0097] As can be seen from the above, the method of the present application performs excellently in various indicators (mIoU, mAP, and Accuracy), especially with significant improvement under low annotation conditions. Experiments show that the method has good segmentation capability for small cracks, fuzzy edges, and low-contrast regions.

[0098] The large building surface crack image segmentation method provided in the application has high precision and high robustness in a low labeling scene by constructing a heterogeneous network structure, introducing a pseudo-label screening mechanism, and designing a consistency and contrast learning module, has significant engineering practical value and popularization prospect, and is suitable for intelligent inspection and structure health assessment scenes of various infrastructures. The method adopted in the application is suitable for automatic identification of concrete structure surface cracks of dams, bridges, tunnels and the like, and still has good crack detection performance and engineering practicability under the condition of insufficient labeled data; has excellent performance and deployment adaptability in engineering scenes with limited labeled samples, and has important engineering application value and popularization prospect.

[0099] The protection scope of the large building surface crack image segmentation method described in the embodiments of the application is not limited to the order of steps listed in the embodiments, and any scheme realized by increasing, replacing or replacing steps of the prior art according to the principles of the application is included in the protection scope of the application.

[0100] The embodiments additionally provide a computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large building surface crack image segmentation method. Figure 1 The large building surface crack image segmentation method.

[0101] In any possible technical detail combination level, the application can be a system, a method and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions stored therein to cause a processor to implement various aspects of the application.

[0102] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punched card or a concave and convex structure in a groove, and any suitable combination of the above. The computer readable storage medium used herein is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (for example, an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0103] The computer readable program here described can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device. Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computing / processing device, partly on the user's computing / processing device, as a stand-alone software package, partly on the user's computing / processing device and partly on a remote computing / processing device or entirely on the remote computing / processing device or server. In the latter scenario, the remote computing / processing device can be connected to the user's computing / processing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0104] The embodiment of the present application further provides a large building surface crack image segmentation system, which can implement the large building surface crack image segmentation method of the present application. However, the implementation device of the large building surface crack image segmentation method of the present application includes but is not limited to the structure of the large building surface crack image segmentation system of the present embodiment. Any prior art structure deformation and replacement according to the principle of the present application is included in the protection scope of the present application.

[0105] The large building surface crack image segmentation system provided by the present embodiment will be described in detail below in combination with the drawings.

[0106] The present embodiment provides a large building surface crack image segmentation system, which comprises: Please refer to Figure 12, shows the principle structure schematic diagram of the large building surface crack image segmentation system described in the application in an embodiment. As shown in Figure 12 The large building surface crack image segmentation system includes an acquisition module 121, a heterogeneous dual-branch network construction module 122, an optimization supervision module 123, a pixel-level contrast module 124, and an output and inference module 125.

[0107] The acquisition module 121 is configured to acquire a building surface original image of a target region to be detected.

[0108] In this embodiment, the building facade in the target region to be detected is divided into several manageable regions (such as by floor, by facade), and each region is numbered to ensure that there is no omission, and fine building region division and planning is performed. Then, appropriate equipment is selected according to the planning and standard image acquisition is performed. Then, the massive images collected are systematically arranged and labeled.

[0109] The heterogeneous dual-branch network construction module 122 is configured to extract a multi-dimensional feature vector of the building surface original image by a heterogeneous network composed of a convolutional neural network and a Transformer decoder branch, and generate an image prediction result; the multi-dimensional feature vector includes a feature map of labeled data and a feature map of unlabeled images; the image prediction result includes a crack segmentation probability feature map and a contrast result.

[0110] In this embodiment, a dual-branch heterogeneous network structure and a multi-scale perturbation design are adopted. That is, by constructing a dual-branch architecture composed of a convolutional neural network and a Transformer decoder, strong perturbation views and weak perturbation views of the same unlabeled image are processed respectively. Through different data enhancement strategies (such as random cropping, color perturbation, CutMix, etc.), the changes of the image in the real environment are simulated, and multi-view feature expression is realized. The two branches share the encoder but use heterogeneous decoders, thereby improving the robustness and feature extraction capability of the model while maintaining the stability of the structure semantics.

[0111] Specifically, based on the building surface original image, a building surface labeled image and an unlabeled image are obtained; based on the building surface labeled image, encoding and decoding processing and feature extraction are performed to obtain a feature map of labeled data; and a multi-scale perturbation strategy is used to predict the unlabeled image to obtain a multi-scale perturbation prediction result.

[0112] The optimization supervision module 123 is configured to dynamically adjust based on the image prediction result, and constrain by using a consistency supervision mechanism to obtain an optimized prediction label.

[0113] In this embodiment, data integration is performed based on the crack segmentation probability feature map to generate an optimized pseudo-label mask map; pixel sampling is performed based on the optimized pseudo-label mask map to obtain a pixel sample set of unlabeled data; and pixels with high confidence prediction are extracted from the pixel sample set of unlabeled data, and the pixels with high confidence prediction are used as optimized prediction labels.

[0114] The pixel-level contrast module 124 is configured to perform pixel-level contrast learning on the multi-dimensional feature vector and the optimized prediction label, so as to realize feature alignment of the feature map of the labeled data and the feature map of the unlabeled image, and obtain a final crack segmentation result.

[0115] In this embodiment, high-confidence pseudo-label regions are extracted according to the multi-dimensional feature vector and the optimized prediction label; anchor point contrast pairs are constructed based on the high-confidence pseudo-label regions; similarity between anchor points and positive sample features and negative sample features is obtained according to the anchor point contrast pairs, and a pixel-level contrast loss is calculated.

[0116] Specifically, the feature map of the labeled data, the feature map of the unlabeled image, and the pixel sample and its pseudo-label obtained by sampling are input into the pixel-level contrast learning module. Among them, the pixel features with true labels in the labeled data feature map are taken as anchor points, the unlabeled pixel features of the corresponding class (crack or background) are taken as positive samples, and the unlabeled pixel features not belonging to the class are taken as negative samples.

[0117] The CPCL module measures the similarity between the anchor point features and the positive and negative sample features, and calculates the pixel-level contrast loss: the similarity between the anchor points and the positive sample features is improved, and the similarity between the anchor points and the negative sample features is reduced. Under this contrast learning constraint, the feature alignment of the labeled data and the unlabeled data is realized, and the discrimination ability of the model for crack and background pixels is enhanced. The above contrast loss, together with the supervision loss and the consistency loss, further optimizes the model parameters, and lays a foundation for the final high-precision crack segmentation output.

[0118] The output and inference module 125 is configured to perform inference based on the final crack segmentation result to obtain a crack segmentation probability map, so as to realize crack detection and evaluation of a building surface.

[0119] In this embodiment, after the above training process, the model of the present application finally takes the output of the CNN decoder as the final result of crack segmentation. In the inference stage, the image to be detected is input into the trained encoder and CNN decoder to directly obtain the crack segmentation probability map. The segmentation result output by the CNN decoder is used for actual crack detection and evaluation, and thus the entire CSCL crack segmentation process is completed.

[0120] In summary, the application combines double-branch consistency learning, pseudo-label optimization strategy and contrastive loss design under uncertainty guidance, which realizes high-precision and high-robustness crack detection capability while ensuring low labeling cost.

[0121] The large building surface crack image segmentation model builds a large building surface crack image segmentation system. Under the condition of extremely limited crack image annotation data, the segmentation performance and actual deployment robustness of the model are effectively improved, which is especially suitable for crack monitoring tasks of large infrastructure such as dams, bridges and tunnels. And the application combines double-branch consistency learning, pseudo-label optimization strategy and contrastive loss design under uncertainty guidance, which realizes high-precision and high-robustness crack detection capability while ensuring low labeling cost.

[0122] It should be noted that the division of each module of the above system is only a logical functional division, and all or part of it can be integrated into a physical entity, or physically separated. And these modules can all be realized in the form of software called by the processing element; all can be realized in the form of hardware; some modules can be realized in the form of software called by the processing element, and some modules can be realized in the form of hardware. For example, the x module can be a separately established processing element, or it can be integrated into a chip of the above system, in addition, it can also be stored in the form of program code in the memory of the above system, and the function of the above x module is called and executed by a processing element of the above system. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capability. In the implementation process, each step of the above method or each module can be completed by integrated logic circuit of hardware in the processor element or instruction in the form of software.

[0123] The above modules can be one or more integrated circuits configured to implement the above methods, for example: one or more Application Specific Integrated Circuits (ASICs), or one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), etc. For another example, when a certain module above is implemented in the form of a processing element scheduling code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor that can invoke code. For another example, these modules can be integrated together to implement in the form of a system-on-a-chip (SOC).

[0124] Referring to Figure 13 , a schematic diagram of an electronic device structure described in the present application is shown. As Figure 13 indicated, the electronic device includes a processor 131 and a memory 132; the memory 132 is configured to store a computer program; the processor 131 is connected with the memory 132 and is configured to execute the computer program stored in the memory 132, so that the large building surface crack image segmentation device executes each step of the large building surface crack image segmentation method as described above.

[0125] Preferably, the memory can include a Random Access Memory (RAM) and can also include a non-volatile memory, such as at least one disk memory.

[0126] The above processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0127] To sum up, the large building surface crack image segmentation method, system and device provided by the application have the following beneficial effects: The large building surface crack image segmentation method provided by the application realizes high-precision and high-robustness crack segmentation capability in a low-labeling scene by constructing a heterogeneous network structure, introducing a pseudo-label screening mechanism, and designing a consistency and contrast learning module, and has significant engineering practical value and popularization prospect, and is suitable for intelligent inspection and structure health assessment scenes of various infrastructures.

[0128] The above embodiments only exemplarily illustrate the principles and effects of the application, and are not used to limit the application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the application. Therefore, all equivalent modifications or changes completed by those skilled in the art without departing from the spirit and technical thought disclosed by the application should be covered by the claims of the application.

Claims

1. A method of crack image segmentation of a large building surface, characterized by, The method comprises the following steps: obtaining an original image of a building surface of a target area to be detected; extracting a multi-dimensional feature vector of the original image of the building surface by a heterogeneous network composed of a convolutional neural network and a Transformer decoder branch, and generating an image prediction result; the multi-dimensional feature vector comprises a feature map of labeled data and a feature map of unlabeled images; the image prediction result comprises a crack segmentation probability feature map and a comparison result; based on the image prediction result, dynamic adjustment is performed and a consistency supervision mechanism is used for constraint to obtain an optimized prediction label; pixel-level comparison learning is performed on the multi-dimensional feature vector and the optimized prediction label to realize alignment of the feature map of the labeled data and the feature map of the unlabeled image, and a final crack segmentation result is obtained; based on the final crack segmentation result, a crack segmentation probability map is obtained through reasoning to realize crack detection and evaluation of the building surface.

2. The crack image segmentation method of a large building surface according to claim 1, characterized by, The multi-dimensional feature vector of the original image of the building surface is extracted by a heterogeneous network composed of a convolutional neural network and a Transformer decoder branch, and comprises the following steps: based on the original image of the building surface, a building surface labeled image and an unlabeled image are obtained; based on the building surface labeled image, encoding and decoding processing and feature extraction are performed to obtain a feature map of labeled data; a multi-scale perturbation strategy is used to predict the unlabeled image to obtain a multi-scale perturbation prediction result.

3. The crack image segmentation method of a large building surface according to claim 2, characterized by, The encoding and decoding processing and feature extraction based on the building surface labeled image to obtain the feature map of the labeled data comprises the following steps: based on the building surface labeled image, a labeled image multi-scale feature is obtained through feature extraction; the image feature extracted by the encoder is input into a CNN decoder, and a pixel-level crack segmentation probability map corresponding to the building surface labeled image is output by the CNN decoder; based on the pixel-level crack segmentation probability map, parameter updating is performed, and the labeled image multi-scale feature is mapped to generate a feature map of labeled data.

4. The crack image segmentation method of a large building surface according to claim 2, characterized by, The multi-scale perturbation strategy comprises a strong perturbation path and a weak perturbation path; the multi-scale perturbation strategy is used to predict the unlabeled image to obtain a multi-scale perturbation prediction result, which comprises the following steps: the unlabeled image is processed through encoder feature extraction and CNN decoder reconstruction by a convolutional neural network to obtain a crack segmentation probability map of the unlabeled image, and a crack segmentation probability feature map is generated; the feature map feature of the unlabeled image is extracted and decoded by a Transformer model to obtain a crack segmentation prediction map as a pseudo label of the unlabeled image; the pseudo label and the crack segmentation probability feature map are compared to constrain the training of the model on the unlabeled image to obtain a comparison result; the crack segmentation probability feature map and the comparison result are used as the multi-scale perturbation prediction result.

5. The crack image segmentation method of a large-scale building surface according to claim 1, characterized by, The dynamic adjustment based on the image prediction result and the constraint by the consistency supervision mechanism to obtain the optimized prediction label comprises the following steps: based on the crack segmentation probability feature map, data integration is performed to generate an optimized pseudo label mask map; Based on the optimized pseudo label mask map, unlabeled pixel sampling is performed to obtain a pixel sample set of unlabeled data; From the pixel sample set of the unlabeled data, high-confidence predicted pixels are extracted, and the high-confidence predicted pixels are used as optimized predicted labels.

6. The crack image segmentation method of a large-scale building surface according to claim 5, characterized by, Based on the crack segmentation probability feature map, data integration is performed to generate an optimized pseudo label mask map, including: Based on the crack segmentation probability feature map, a fused average probability map is obtained; The predicted entropy value of each pixel in the fused average probability map is calculated; According to the predicted entropy value of each pixel, dynamic threshold screening is performed to obtain an optimized pseudo label mask map.

7. The crack image segmentation method of a large-scale building surface according to claim 5, characterized by, Based on the optimized pseudo label mask map, unlabeled pixel sampling is performed to obtain a pixel sample set of unlabeled data, including: In the optimized pseudo label mask map, a plurality of samples are extracted to construct a pixel sample set of unlabeled data; the pixel sample set of unlabeled data includes positive samples and negative samples; The positive samples are pixel features in the crack area indicated by the mask, and the negative samples are pixel features sampled from the background area outside the mask.

8. The crack image segmentation method of a large-scale building surface according to claim 1, characterized by, The multi-dimensional feature vector and the optimized predicted label are subjected to pixel-level contrast learning to realize feature alignment of the labeled data feature map and the unlabeled image feature, and a final crack segmentation result is obtained, including: According to the multi-dimensional feature vector and the optimized predicted label, a high-confidence pseudo label region is extracted; Based on the high-confidence pseudo label region, anchor point contrast comparison is constructed; According to the anchor point contrast comparison, the similarity between the anchor point and the positive sample feature and the negative sample feature is obtained, and a pixel-level contrast loss is calculated; the pixel-level contrast loss includes a supervision loss, a consistency loss, and a contrast loss.

9. A large building surface crack image segmentation system, characterized by, including: An acquisition module is configured to acquire a building surface original image of a target region to be detected; A heterogeneous dual-branch network construction module is configured to extract a multi-dimensional feature vector of the building surface original image and generate an image prediction result by using a heterogeneous network composed of a convolutional neural network and a Transformer decoder branch; The multi-dimensional feature vector includes a labeled data feature map and an unlabeled image feature map; the image prediction result includes a crack segmentation probability feature map and a contrast result; An optimization supervision module is configured to dynamically adjust based on the image prediction result and constrain by using a consistency supervision mechanism to obtain an optimized predicted label; A pixel-level contrast module is configured to perform pixel-level contrast learning on the multi-dimensional feature vector and the optimized predicted label to realize feature alignment of the labeled data feature map and the unlabeled image feature, and obtain a final crack segmentation result; An output and inference module is configured to perform inference based on the final crack segmentation result, obtain a crack segmentation probability map, and realize crack detection and evaluation of a building surface.

10. An electronic device, comprising: including: A memory is configured to store a computer program; A processor is configured to execute the computer program stored in the memory, so that the electronic device performs the large building surface crack image segmentation method according to any one of claims 1 to 8.

Images