Concrete crack unsupervised segmentation method and system based on CLIP large model and pseudo mask self-purification

Abstract

The invention provides a concrete crack unsupervised segmentation method and system based on a CLIP large model and pseudo mask self-purification. The method comprises the following steps: firstly, generating an image-level pseudo tag by using a CLIP model; a class activation mask is generated through layer class activation mapping; sauvola threshold segmentation is applied to the class activation mask foreground area to generate a four-value threshold mask; based on a class activation mask and a four-value threshold mask, automatically generating a frame, a background point and a foreground point prompt, and inputting the prompt and an original image into an SAM2 model to obtain an SAM2 segmentation mask; fusing the class activation mask, the threshold mask and the SAM2 segmentation mask to obtain a four-valued pseudo mask comprising a high-confidence foreground, a medium-confidence foreground, an uncertain region and a background region; taking the four-valued pseudo mask as a supervision signal to train a segmentation model, and dynamically updating a pseudo tag in training to realize self-purification of the pseudo tag and iterative optimization of model performance; and finally, utilizing the trained classification model and segmentation model to automatically identify crack images in sequence.

Description

TECHNICAL FIELD

[0001] The application provides a concrete crack unsupervised segmentation method and system based on a CLIP large model and pseudo mask self-purification, and relates to the technical fields of image semantic segmentation and concrete crack identification. BACKGROUND

[0002] During the long-term service of civil engineering structures such as roads, bridges, tunnels and dams, various surface diseases inevitably occur in concrete materials due to factors such as fatigue load, environmental erosion, temperature change and chemical corrosion. As one of the most typical and most common early diseases, cracks not only affect the appearance of the structure, but also may accelerate the invasion of moisture and corrosive media, leading to steel corrosion and protective layer spalling, which seriously threatens the durability and bearing safety of the structure. Timely detection and evaluation of crack status is a key link for implementing predictive maintenance and prolonging the service life of infrastructure.

[0003] Traditional crack detection mainly relies on manual visual inspection, which has problems such as low efficiency, strong subjectivity, high missed detection rate and high safety risk. Traditional image processing methods include edge detection, threshold segmentation and morphological processing, etc. These methods are extremely sensitive to changes in illumination, shadow interference and surface texture noise, and have poor robustness, and are prone to false detection or missed detection in complex actual scenes. In recent years, semantic segmentation based on deep learning has significantly improved the detection accuracy and automation level, but its performance depends on a large amount of pixel-level labeled data, which is time-consuming and labor-intensive for manual labeling, and there are subjective differences between different labelers, which restricts the wide application of this method.

[0004] Weakly supervised and unsupervised semantic segmentation methods can reduce the dependence on labeled data, but the pseudo masks generated by existing technologies have gaps with real labels, and the model is prone to learn noise information in the pseudo masks during the training process, resulting in blurred boundaries of the segmentation results and high missed detection rate of small cracks, which is difficult to meet the requirements of precision and reliability in engineering applications. Therefore, there is an urgent need for an unsupervised crack segmentation method that does not require manual labeling, can generate high-quality pseudo masks and has high precision. SUMMARY

[0005] Therefore, in order to make up for the shortcomings of the prior art, the application provides a concrete crack unsupervised segmentation method and system based on a CLIP large model and pseudo mask self-purification, which can accurately identify cracks on the surface of concrete without any manual labeling.

[0006] The application provides a concrete crack unsupervised segmentation method and system based on a CLIP large model and pseudo mask self-purification, which includes the following contents:

[0007] The application provides a concrete crack unsupervised segmentation method based on a CLIP large model and pseudo mask self-purification.

[0008] Step S1: Obtain a concrete surface image dataset, construct crack and normal text prompts, and use a CLIP model to perform image-level classification on input concrete surface images to generate image-level pseudo labels.

[0009] Step S2: Train a ResNet50 classification model based on the image-level pseudo labels.

[0010] Step S3: Use LayerCAM technology to generate class activation mapping of the trained classification model, and convert it into a binary class activation mask containing foreground and background regions through dynamic threshold processing, to realize coarse positioning of the crack.

[0011] Step S4: Apply Sauvola adaptive threshold segmentation in the foreground region of the binary class activation mask to generate a four-value threshold mask containing high-confidence foreground, medium-confidence foreground, low-confidence foreground and background regions, and combine connected component analysis to remove noise regions that do not meet the geometric characteristics of the crack, to realize more precise positioning of the crack region.

[0012] Step S5: Generate point and box prompts according to the binary class activation mask and the four-value threshold mask, generate multiple candidate prompt combinations, input the prompts and the original concrete surface image into a SAM2 model, and use a multiple quality filtering mechanism to screen the segmentation results that meet the conditions, and output a SAM2 segmentation mask.

[0013] Step S6: Process the binary class activation mask, the four-value threshold mask and the SAM2 segmentation mask through a multi-source fusion strategy to generate a final four-value pseudo mask, wherein the four-value pseudo mask includes a high-confidence foreground region, a medium-confidence foreground region, an uncertain region and a background region.

[0014] Step S7: Use the four-value pseudo mask in step S6 as a supervision signal to train a U-Net segmentation model; the training uses a weighted binary cross-entropy loss function, and the pixels in the uncertain region are not involved in the loss calculation; during the training process, a data enhancement strategy is applied, including random scaling, cropping, horizontal flipping and color disturbance; and a dynamic self-purification mechanism is combined to update the pseudo mask until a predetermined number of iterations is reached or the proportion of pixels in the uncertain region changes below a preset threshold, and the trained segmentation model is saved.

[0015] Step S8: Use the classification model trained in step S2 to classify the concrete surface image; if it is determined that there is a crack, use the segmentation model trained in step S7 to perform pixel-level segmentation on the image containing the crack.

[0016] Further, step S1 comprises the following contents:

[0017] The cosine similarity between the image encoding and the text encoding is calculated, and after normalization by the Softmax function, the posterior probability of each category is obtained, and the category with the highest probability is taken as the image-level pseudo label.

[0018] Further, step S3 comprises the following contents:

[0019] The coarse positioning process of the crack in step S3 comprises: selecting the last residual block of the ResNet50 classification model as the feature extraction layer, calculating the gradient of the crack category prediction score with respect to the feature map of the layer; generating a class activation heat map by gradient weighted fusion of the feature map, and taking the α percentile of the pixel values in the class activation heat map in ascending order as the threshold T a , the region with response value greater than T a is marked as foreground, and the region with response value less than T a is marked as background.

[0020] Further, step S4 comprises the following contents:

[0021] Step S41: the Sauvola adaptive threshold comprises an algorithm for dynamically determining the segmentation threshold based on the local window mean and standard deviation of pixels, wherein the Sauvola adaptive threshold is defined as:

[0022]

[0023] Wherein μ(x,y) and σ(x,y) are the mean and standard deviation of the local window centered at pixel (x,y), R is the standard deviation range, and k is the control parameter;

[0024] Step S42: construct a three-layer threshold group T H (x,y), T M (x,y), T L (x,y), whose corresponding control parameters are k H , k M , k L , and satisfy T H (x,y)<T M (x,y)<T L (x,y) in the image domain;

[0025] Step S43: define three foreground sets:

[0026] H th = {(x,y)|I(x,y)<T H (x,y)};

[0027] M th ={(x,y)|I(x,y)<T M (x,y)};

[0028] L th ={(x,y)|L(x,y)<T L (x,y)};

[0029] where I(x,y) is the pixel gray value, and satisfies the strict hierarchical inclusion relationship According to the three-layer results and the binary class activation mask, the final four-value threshold mask is constructed: high confidence foreground is H th , medium confidence foreground is (M th -H th ), where (M th -H th ) represents the elements in set M th but not in set H th , low confidence foreground is (L th -H th -M th ), where (L th -H th -M th ) represents the elements in set L th but not in set H th and M th , and the background is the remaining pixels, the background is the remaining pixels.

[0030] Step S44: The connected domain analysis is based on area and aspect ratio to perform double threshold filtering, and connected domains with an area less than A min and connected domains with an aspect ratio lower than a set threshold are removed, and the aspect ratio calculation formula is as follows:

[0031]

[0032] where L ske1 is the connected domain skeleton length, and A is the connected domain area.

[0033] Further, step S5 includes the following contents:

[0034] In step S5, the prompt combination includes a box, a background, and a foreground point prompt; the box prompt extracts the minimum circumscribed rectangle from the foreground region of the class activation binary mask; the background points are pre-sampled from the background region of the class activation mask by farthest point sampling; the foreground point prompt is preferentially pre-sampled from the high-confidence foreground region of the four-value threshold mask, and if insufficient, is sequentially supplemented from the medium-confidence and low-confidence foreground regions, and the hierarchical sampling strategy ensures that the foreground point prompt input into the SAM2 model is maximally derived from the most reliable candidate region, thereby reducing the number of attempts of invalid prompts.

[0035] The multiple quality filtering mechanism includes the following conditions: the intersection of the foreground region of the SAM2 segmentation mask and the foreground region of the class activation mask accounts for no less than a threshold τ of the foreground region of the SAM2 segmentation mask c ; the foreground area of the SAM2 segmentation mask is no more than τ of the foreground area of the class activation mask ; the aspect ratio of the SAM2 segmentation mask is no less than τ of the aspect ratio of the class activation mask s ; the total number of connected domains in the SAM2 segmentation mask is greater than 1 but no more than τ n ; only when the candidate SAM2 segmentation mask simultaneously satisfies all the above conditions, is it retained as a valid output.

[0036] Further, step S5 further includes the following content:

[0037] In step S5, the high-confidence foreground region is the intersection of the SAM2 segmentation mask and the high-confidence foreground region of the four-value threshold mask; the medium-confidence foreground region is the part of the SAM2 segmentation mask that is not covered by the high-confidence region; the uncertain region is the part of the foreground region of the binary class activation mask that is not covered by the high-confidence foreground region and the medium-confidence foreground region; and the background region is the remaining part of the image other than the above foreground and uncertain regions.

[0038] Further, step S7 further includes the following content:

[0039] In step S7, the loss function of the weighted binary cross-entropy is defined as:

[0040]

[0041] CE(p i ,y i )=-[y i log(p i )+(1-y i )log(1-p i )];

[0042] wherein the sets H, M, B, and U represent the pixel sets of the high-confidence foreground, the medium-confidence foreground, the background, and the uncertain region, respectively; NH N M N B p represents the number of pixels in the corresponding set; for pixels i∈U, cross-entropy loss is not calculated and they do not participate in gradient backpropagation; i Let y be the probability that the model predicts pixel i as foreground. i ∈[0,1] is a pseudo-label; w H w M w B These are the loss weighting coefficients for the high-confidence prospect, medium-confidence prospect, and background regions, respectively, CE(p i ,y i () represents the classification cross-entropy loss function, used to calculate the prediction probability p of the segmentation model for a single pixel. i With pseudo-labels y i The differences between them.

[0043] Furthermore, step S7 also includes the following:

[0044] In step S7, the dynamic label update mechanism is executed on a fixed iteration cycle, with a fixed iteration interval divided into a training segment. At the end of each training segment, the model with the highest average recall rate on high-confidence foreground and background in the training set is saved as the optimal model for the current stage. The optimal model is used to update the current multi-level pseudomask. The update mechanism includes: for pixels in the current uncertain region that are predicted as background by the optimal model, when their background probability reaches a set threshold, they are updated as background by a certain proportion, gradually reducing the noise region in the initial pseudolabel; when the pixel change ratio in the uncertain region in this round is lower than a preset threshold, foreground denoising and foreground restoration operations are triggered. For pixels in the medium-confidence foreground region that are predicted as background by the optimal model and whose probability reaches the set threshold, they are downgraded to uncertain regions using a small batch method; at the same time, for pixels in the uncertain region of the initial pseudomask that are predicted as foreground by the optimal model and whose probability reaches the set threshold, they are marked as medium-confidence foreground regions, forming an update closed loop; the high-confidence foreground and background regions of the initial pseudomask do not participate in the update throughout the entire process to maintain the overall stability of the supervision signal.

[0045] According to a second aspect of the present invention, the present invention proposes an unsupervised concrete crack segmentation system based on CLIP large model and pseudo-mask self-purification, comprising an electronic device, wherein the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements an unsupervised concrete crack segmentation method based on CLIP large model and pseudo-mask self-purification as described in any one of the present invention.

[0046] According to a third aspect of the present invention, the present invention proposes an unsupervised concrete crack segmentation system based on CLIP large model and pseudo-mask self-purification, comprising a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements an unsupervised concrete crack segmentation method based on CLIP large model and pseudo-mask self-purification as described in any one of the present invention.

[0047] The present invention has the following advantages:

[0048] 1. Without any manual annotation, the CLIP zero-shot classification and multi-stage pseudo-label generation strategy enables the fully automated construction of high-quality supervision signals, significantly reducing data preparation and time costs, and making it suitable for large-scale engineering application scenarios.

[0049] 2. By using indicators such as the aspect ratio, area, and number of connected components to filter SAM2 candidate masks, the missegmentation caused by ambiguity in the prompts is effectively suppressed, making the output results more consistent with the actual crack morphology.

[0050] 3. By integrating LayerCAM's semantic prior, Sauvola's local contrast response, and SAM2's fine boundary segmentation capabilities, the generated multi-layer pseudomask not only ensures the accuracy of the foreground region but also retains potential effective information through uncertain regions, effectively mitigating noise interference and missed detection problems.

[0051] 4. Compared with traditional binary pseudomask generation methods, the method described in this invention adopts a multi-level pseudomask supervised training mechanism. During the training process, the pseudo-labels are iteratively optimized through a dynamic update mechanism, avoiding the model learning noise information in the pseudomask, significantly reducing the false detection rate of small cracks and the false detection rate of background, and improving the completeness and accuracy of the segmentation results. Attached Figure Description

[0052] Figure 1 This is a flowchart of the present invention.

[0053] Figure 2 The visualization results of the class activation mask of the present invention are generated.

[0054] Figure 3 This is the SAM2 model segmentation process of the present invention.

[0055] Figure 4 This is a demonstration of the results before and after SAM2 segmentation mask filtering according to the present invention. Detailed Implementation

[0056] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0057] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0058] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0059] This invention includes the following:

[0060] CLIP (Contrastive Language-Image Pre-Training) is a visual language pre-training model consisting of an image encoder and a text encoder.

[0061] LayerCAM (Layer-wise Class Activation Mapping) is a gradient-guided method for generating class activation maps. It generates high-resolution heatmaps by utilizing gradient information from feature maps of specific layers in a classification model. These heatmaps are used to accurately locate and visualize the model's regions of interest for specific classes.

[0062] Sauvola adaptive thresholding is an adaptive binarization method based on local image statistical features. It calculates the local mean and standard deviation through a neighborhood window centered on a pixel and dynamically adjusts the threshold to cope with uneven lighting and texture changes.

[0063] SAM2 (Segment Anything Model 2) is a general image segmentation model that supports various interaction methods such as point hints and bounding box hints, and can perform zero-shot segmentation based on hints.

[0064] like Figures 1 to 4 As shown, this invention proposes an unsupervised method and system for concrete crack segmentation based on CLIP large model and pseudo-mask self-purification, characterized by the following:

[0065] like Figure 1As shown, in one embodiment of the present invention, step S1 is performed to acquire a dataset of RGB images of a concrete surface, including images of a normal surface and images with cracks. Two types of text prompts are constructed: a prompt for cracks is "a photo of a concrete surface with cracks", and a prompt for normal surfaces is "a photo of a clean concrete surface". Both types of prompts are input into the CLIP text encoder to obtain a comparison vector. Each image is normalized and then fed into the image encoder. Cosine similarity matching is performed with the output of the aforementioned text encoder, and Softmax normalization is applied to obtain the probability distribution of belonging to "cracks" or "no cracks". The category with the highest probability is taken as the pseudo-label of the sample, resulting in a dataset D1 containing image-level pseudo-labels and the original images.

[0066] In one embodiment of the present invention, step S2 is performed to augment dataset D1 to simulate complex on-site conditions. Two augmentation methods are employed: black rectangular box occlusion and shadow interference. Occlusion augmentation uses black rectangular boxes of random size and position to occlude the image, while shadow interference augmentation overlays random polygonal shadow regions onto the image and applies Gaussian blur to simulate non-uniform lighting conditions. The augmented dataset D2 is then obtained and used for supervised training of the ResNet50 classification model. Model optimization uses the stochastic gradient descent algorithm, with an initial learning rate of 0.001, a momentum parameter of 0.9, a batch size of 16, and a training epoch of 3.

[0067] In one embodiment of the present invention, step S3 is executed, loading the trained ResNet50 classification model and using LayerCAM technology to generate a class activation map reflecting the model's regions of interest. For each original concrete crack image to be processed, the system adjusts it to a uniform standard size and performs standardization preprocessing before inputting it into the classification model to obtain the category prediction result. For images determined by the model to be without cracks, the system directly generates a completely black mask image; for images identified as containing cracks, further calculation of the class activation map is performed. When generating the class activation map, the system uses the last convolutional layer of the model as the target layer and generates a heatmap using LayerCAM technology. The heatmap visually displays the image regions that the model is most interested in during the classification decision process, corresponding to the potential locations of cracks. The class activation map is converted into a binary mask using a dynamic threshold. Figure 2 The class activation map and the transformed binary mask are shown. The dynamic thresholding process uses the percentile thresholding method, which sorts all pixel values ​​greater than zero in the normalized class activation heatmap in ascending order, and then takes the a-th percentile as the dynamic threshold T. a When 'a' is set to 70, 70% of the effective response values ​​in the thermogram are lower than T. 70 30% higher than T70 The critical point is to compare the response values ​​of all pixels in the heatmap with T. 70 The response values ​​are compared and are greater than or equal to T. 70 The region marked as foreground has a response value less than T. 70 The area is then marked as the background.

[0068] In one embodiment of the present invention, step S4 is performed, employing a multi-level Sauvola adaptive thresholding algorithm within the foreground region marked by the class activation mask. The Sauvola adaptive threshold is defined per pixel as follows:

[0069]

[0070] Where μ(x,y) and σ(x,y) are the mean and standard deviation of the local window centered at pixel (x,y), respectively, R is the standard deviation range, and k is the control parameter;

[0071] Based on this local threshold, a three-layer threshold group T is constructed. H (x,y),T M (x,y),T L (x, y), whose corresponding control parameters are k H =0.3,k M =0.2,k L =0.1, and satisfies T in the image domain H (x,y) <T M (x,y) <T L (x, y). Therefore, three types of foreground sets are defined:

[0072] H th ={(x,y)|I(x,y) <T H (x,y)};

[0073] M th ={(x,y)|I(x,y) <T M (x,y)};

[0074] L th ={(x,y)|L(x,y) <T L (x,y)};

[0075] Where I(x,y) is the pixel grayscale value, and satisfies a strict hierarchical containment relationship. Based on the three-layer results and the binary activation mask, the final four-value threshold mask is constructed: the high-confidence foreground is H. th Zhongzhixin's prospects are (M) th -H th ), of which (M th -H th) represents set M th There is a set H th For elements not present in the given information, the low-confidence prospect is (L). th -H th -M th ), of which (L th -H th -M th Show set L th There is a set H th and M th Elements not present in the image are represented by the remaining pixels in the background. Next, the area of ​​the largest connected component in the threshold mask is calculated, dynamically determining the area threshold as the larger of the area of ​​the largest connected component divided by 10.0 and 50.0. For connected components with an area smaller than this threshold, if the aspect ratio of their bounding rectangle is less than 2, they are directly removed. If the aspect ratio of their bounding rectangle is greater than or equal to 2 but the aspect ratio of their skeleton is less than 4, they are also filtered. The skeleton aspect ratio is defined as the ratio of the square of the skeleton length to the area of ​​the connected component, and its calculation formula is:

[0076]

[0077] Where L skel Let A be the length of the connected component skeleton and A be the area of ​​the connected component. Only connected components that simultaneously satisfy the area and shape structure constraints are retained to improve the reliability of foreground point prompts.

[0078] Execute step S5 to generate a cue for SAM2 based on the class activation mask and the multi-level threshold mask. Figure 3 The SAM2 segmentation process of this invention is demonstrated. The minimum bounding rectangle of the foreground region of the class-activation mask is extracted as the initial bounding box cue. In the background region of the class-activation mask, a farthest-point sampling strategy is used to uniformly select 8-13 background points, resulting in 6 sets of background point cue combinations. Foreground cue points are presampled multiple times in a four-value threshold mask according to confidence levels, with the sampling order being high confidence, medium confidence, and low confidence, ensuring that the most reliable crack pixels are prioritized for cue generation; if the number of confidence points in the previous layer is insufficient, it is automatically supplemented from the next layer. Six foreground points are presampled; each time a cue is constructed, these six foreground points are combined with 1-3 points, resulting in 41 sets of foreground cue combinations. Two types of cue combinations are constructed: the first type uses only bounding box cue and background point cue, totaling 6 sets; the second type includes bounding box cue, background point cue, and foreground point cue simultaneously, totaling 6 × 41 = 246 sets; a total of 252 potential cue combinations are obtained. For each cue combination, SAM2 generates a candidate segmentation mask and filters them one by one based on multiple geometric metrics, including coverage, area range, number of connected components, and aspect ratio. First, the coverage metric requires that the intersection ratio of the foreground region of the SAM2 mask and the foreground region of the CAM exceeds a preset threshold.

[0079]

[0080] Among them, M sam It is the foreground mask output by SAM2, M cam This is the foreground region of the CAM mask. This constraint ensures consistency between the segmentation results and the coarse CAM localization, preventing the introduction of global offsets or irrelevant regions. Considering the elongated nature of the crack, the area index requires the SAM2 foreground area to be significantly smaller than the CAM foreground region to avoid over-segmentation caused by ambiguity in the hints.

[0081]

[0082] The CAM mask is a coarse localization mask that contains a lot of background noise. The aspect ratio requirement is that the aspect ratio of the SAM2 mask is not less than a certain multiple of the aspect ratio of the CAM mask.

[0083] S(M sam )≥S(M cam )×2

[0084] The aspect ratio function is defined as follows:

[0085]

[0086] L skeleton (M) represents the skeleton length of the mask, and Area(M) represents the pixel area of ​​the mask. The multi-filtering mechanism uses the CAM mask as a dynamic reference to adapt to the diversity of different crack shapes. At the same time, to avoid generating a large number of noisy foregrounds or fragmented regions, the number of connected components is set to be no less than 1 and no more than 5 to ensure the topological integrity and coherence of the segmentation results. A candidate mask is only considered a valid output if it meets all of the above conditions. If it does not meet the conditions, the system will automatically try the next set of prompt combinations until the set number of prompt combinations has been traversed or a valid mask that meets the constraints is successfully obtained. Figure 4 This shows a comparison of segmentation results for some images before and after filtering. Figure 4 In the image, red pentagrams represent background points, green pentagrams represent foreground points, green borders represent framed points, and blue areas represent segmented areas.

[0087] In one embodiment of the present invention, step S6 is performed to fuse the class activation mask, the four-value threshold mask, and the SAM2 segmentation mask to generate a final pixel-level pseudo-mask. The high-confidence foreground region is the intersection of the high-confidence regions of the SAM2 segmentation mask and the four-value threshold mask; the medium-confidence foreground region is the independent portion of the SAM2 segmentation mask not covered by the high-confidence region; the uncertain region is the remaining portion of the foreground region of the class activation mask not covered by the high-confidence and medium-confidence foreground regions; and the background region is all remaining portions of the image except for the foreground and uncertain regions. The final pseudo-mask uses four-value encoding, corresponding to the four types of regions: high-confidence foreground, medium-confidence foreground, uncertain region, and background, respectively, resulting in a dataset D3 containing pixel-level pseudo-labels and the original image.

[0088] In one embodiment of the present invention, step S7 is performed to train the U-Net segmentation model using dataset D3. The model input is the original image, and the output is a binary segmentation map, with supervision using a pseudo-mask. The loss function is a weighted binary cross-entropy loss.

[0089]

[0090] CE(p i ,y i )=-[y i log(p i )+(1-y i log(1-p) i )];

[0091] Where sets H, M, B, and U represent the pixel sets of high-confidence foreground, medium-confidence foreground, background, and uncertain region, respectively; N H N M N B p represents the number of pixels in the corresponding set; for pixels i∈U, cross-entropy loss is not calculated and they do not participate in gradient backpropagation; i Let y be the probability that the model predicts pixel i as foreground. i ∈[0,1] is a pseudo-label; w H w M w B These are the loss weighting coefficients for the high-confidence prospect, medium-confidence prospect, and background regions, respectively, CE(p i ,y i () represents the classification cross-entropy loss function, used to calculate the prediction probability p of the segmentation model for a single pixel. i With pseudo-labels y i The differences between them.

[0092] In one embodiment of the present invention, w H =1.2,w M =0.7,w B=1.0 represents the loss weight coefficients for high-confidence foreground, medium-confidence foreground, and background regions, respectively; uncertain regions are not included in the loss calculation. The optimizer uses Adam with an initial learning rate of 0.001, a batch size of 16, and a total of 10,000 iterations. Data augmentation is performed during training using methods such as random scaling, random cropping, horizontal flipping, and color distortion. The segmentation model training process adopts a segmented iterative strategy, with each training segment consisting of 500 iterations, achieving progressive self-purification of the pseudomask. Within each training segment, the model is evaluated on the training set every 100 iterations, and the model with the highest average recall on the high-confidence foreground and background is saved as the optimal model for that stage. Based on the prediction results of this optimal model, the current pseudomask is dynamically updated, specifically including the following process: After each training segment, the total number of pixels marked as uncertain in the current pseudomask is first counted, and the proportion of each candidate pixel is calculated using this total as the denominator. The pixels are sorted from highest to lowest background probability. The initial training segment update limit is set to 10% of the number of uncertain pixels. In each subsequent round, this limit increases by 10 percentage points from the previous round. The maximum number of pixels updated to background in each round cannot exceed 90% of the current uncertain pixels. Only when the background probability of a candidate pixel is higher than a hard threshold of 0.9 will it be labeled as background. If the uncertain region update ratio in this training segment is less than 1% (where the uncertain region update ratio is the proportion of pixels updated to background in this round out of the total number of pixels in the image), then most of the original uncertain region has been updated to background, triggering foreground denoising and initiating a foreground restoration strategy. Among all pixels in the medium-confidence foreground, pixels with a background probability higher than 0.9 are selected and sorted in descending order of probability. 10% of these candidate pixels are then denoised and updated to represent uncertain regions. This allows the model to re-evaluate these easily confused regions during subsequent training. A small-batch, gradual downgrading approach is used to avoid model instability caused by a large-scale, one-time modification. Simultaneously, a foreground restoration strategy is employed to restore potentially mistakenly deleted crack pixels. Restoration candidate pixels must meet the following conditions: the current label is background or uncertain region, but in the initial pseudomask, it belongs to the foreground or uncertain region, and the model's predicted foreground probability is higher than 0.9. The restoration ratio is 5% of the candidate pixels. It should be noted that high-confidence foreground and background regions in the initial pseudomask are not updated throughout the entire process to maintain the overall stability of the supervision signal. The updated multi-level pseudomask from each training segment is used for model training in the next training segment, achieving co-optimization of supervision signal and model performance. Training ends after a specified number of iterations, and the U-Net segmentation model is saved.

[0093] In one embodiment of the present invention, step S8 is executed: for a newly acquired concrete surface image, the trained ResNet50 classification model is input to determine whether cracks exist. If the image is determined to contain cracks, it is input into the trained U-Net segmentation model, and a binary crack segmentation result is output.

[0094] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. An unsupervised method for concrete crack segmentation based on CLIP large model and pseudo-mask self-purification, characterized in that, Includes the following: Step S1: Obtain a concrete surface image dataset, construct text prompts for cracks and normal surfaces, use the CLIP model to perform image-level classification of cracked and crack-free concrete surface images, and generate image-level pseudo-labels. Step S2: Train a ResNet50 classification model based on the image-level pseudo-labels; Step S3: Use LayerCAM technology to generate a class activation map for the trained classification model, and use dynamic thresholding to convert it into a binary class activation mask containing the foreground and background regions to achieve coarse localization of the crack. Step S4: Apply Sauvola adaptive threshold segmentation to the foreground region of the binary activation mask to generate a four-value threshold mask containing high-confidence foreground, medium-confidence foreground, low-confidence foreground and background regions. Combine this with connected component analysis to remove noise regions that do not conform to the geometric features of the crack, and perform more precise localization of the crack region. Step S5: Generate point hints and bounding box hints based on the binary activation mask and the four-value threshold mask, generate multiple sets of candidate hint combinations, input the hints and the original concrete surface image into the SAM2 model, and use a multi-level quality filtering mechanism to filter the segmentation results that meet the conditions, and output the SAM2 segmentation mask. Step S6: The binary activation mask, quaternary threshold mask, and SAM2 segmentation mask are processed by a multi-source fusion strategy to generate the final quaternary pseudo-mask, which includes a high-confidence foreground region, a medium-confidence foreground region, an uncertain region, and a background region. Step S7: Train the U-Net segmentation model using the four-valued pseudo-mask described in Step S6 as the supervision signal; the training adopts a weighted binary classification cross-entropy loss function, and pixels in uncertain regions do not participate in the loss calculation; data augmentation strategies are applied during the training process, including random scaling, cropping, horizontal flipping, and color perturbation; The pseudomask is updated in conjunction with a dynamic self-purification mechanism until the predetermined number of iterations is reached or the pixel change ratio of the uncertain region is lower than a preset threshold, at which point the update is terminated and the trained segmentation model is saved. Step S8: Use the classification model trained in step S2 to classify the concrete surface image; if cracks are determined to exist, use the segmentation model trained in step S7 to perform pixel-level segmentation on the image containing cracks.

2. The unsupervised segmentation method for concrete cracks based on CLIP large model and pseudo-mask self-purification as described in claim 1, characterized in that, Step S1 includes the following: By calculating the cosine similarity between the image encoding and the text encoding, and normalizing it using the Softmax function, the posterior probability of each category is obtained, and the category with the highest probability is used as the image-level pseudo-label.

3. The unsupervised segmentation method for concrete cracks based on CLIP large model and pseudo-mask self-purification as described in claim 1, characterized in that, Step S3 includes the following: The coarse localization process for cracks described in step S3 includes: selecting the last residual block of the ResNet50 classification model as the feature extraction layer, calculating the gradient of the crack category prediction score relative to the feature map of that layer; generating a class activation heatmap by gradient-weighted fusion of feature maps, and taking the α-th percentile of the pixel values ​​in the class activation heatmap after sorting them in ascending order as the threshold T. a The response value is greater than T. a The region marked as foreground is smaller than T. a The area is marked as the background.

4. The unsupervised segmentation method for concrete cracks based on CLIP large model and pseudo-mask self-purification as described in claim 1, characterized in that, Step S4 includes the following: Step S41: The Sauvola adaptive threshold includes an algorithm that dynamically determines the segmentation threshold based on the mean and standard deviation of the local window of a pixel, wherein the Sauvola adaptive threshold is defined per pixel as follows: Where μ(x,y) and σ(x,y) are the mean and standard deviation of the local window centered at pixel (x,y), respectively, R is the standard deviation range, and k is the control parameter; Step S42: Construct a three-layer threshold group T H (x,y),T M (x,y),T L (x, y), whose corresponding control parameters are k H ,k M ,k L And satisfy T in the image domain H (x,y) <T M (x,y) <T L (x,y); Step S43: Define three types of foreground sets: H th ={(x,y)|I(x,y)<T H (x,y)}; M th ={(x,y)|I(x,y)<T M (x,y)}; L th ={(x,y)|L(x,y) <T L (x,y)}; Where I(x,y) is the pixel grayscale value, and satisfies a strict hierarchical containment relationship. Based on the three-layer results and the binary activation mask, the final four-value threshold mask is constructed: the high-confidence foreground is H. th Zhongzhixin's prospects are (M) th -H th ), of which (M th -H th ) represents set M th There is a set H th For elements not present in the given information, the low-confidence prospect is (L). th -H th -M th ), of which (L th -H th -M th Show set L th There is a set H th and M th For elements not present in the text, the background consists of the remaining pixels; Step S44: The connected component analysis performs dual threshold filtering based on area and aspect ratio, eliminating components with an area smaller than A. min Connected components and connected components with aspect ratios below a set threshold are defined by the following formula: Where L ske1 Let A be the length of the connected component skeleton and A be the area of ​​the connected component.

5. The unsupervised segmentation method for concrete cracks based on CLIP large model and pseudo-mask self-purification as described in claim 1, characterized in that, Step S5 includes the following: In step S5, the cue combination includes a bounding box, a background, and a foreground cue. The bounding box cue extracts the minimum bounding rectangle from the foreground region of the class-activation binary mask. The background points are presampled from the background region of the class-activation mask using farthest point sampling. The foreground cue prioritizes presampling multiple points from the high-confidence foreground region of the four-valued threshold mask, and if insufficient, it is supplemented from the medium-confidence and low-confidence foreground regions in sequence. This hierarchical sampling strategy ensures that the foreground cue input to the SAM2 model is derived from the most reliable candidate regions to the greatest extent possible, reducing the number of attempts for invalid cue. The multi-level quality filtering mechanism includes the following condition: the intersection of the foreground region of the SAM2 segmentation mask and the foreground region of the class activation mask occupies a proportion of the foreground region of the SAM2 segmentation mask that is not less than a threshold τ. c The foreground area of ​​the SAM2 segmentation mask does not exceed the foreground area of ​​the class activation mask. The aspect ratio of the SAM2 segmentation mask is not less than τ of the aspect ratio of the class activation mask. s The total number of connected components in the SAM2 segmentation mask is greater than 1, but does not exceed τ. n A candidate SAM2 segmentation mask is retained as a valid output only if it satisfies all of the above conditions.

6. The unsupervised segmentation method for concrete cracks based on CLIP large model and pseudo-mask self-purification as described in claim 5, characterized in that, Step S5 also includes the following: In step S5, the high-confidence foreground region is the intersection of the SAM2 segmentation mask and the high-confidence foreground region of the four-value threshold mask; the medium-confidence foreground region is the part of the SAM2 segmentation mask that is not covered by the high-confidence region; the uncertain region is the part of the foreground region of the binary activation mask that is not covered by the high-confidence foreground region and the medium-confidence foreground region; and the background region is the remaining part of the image excluding the foreground and uncertain regions mentioned above.

7. The unsupervised segmentation method for concrete cracks based on CLIP large model and pseudo-mask self-purification as described in claim 1, characterized in that, Step S7 also includes the following: The loss function for the weighted binary cross-entropy in step S7 is defined as follows: CE(p i ,y i )=-[y i l(p i )+(1-y i )log(1-p i )]; Where sets H, M, B, and U represent the pixel sets of high-confidence foreground, medium-confidence foreground, background, and uncertain region, respectively; N H N M N B p represents the number of pixels in the corresponding set; for pixels i∈U, cross-entropy loss is not calculated and they do not participate in gradient backpropagation; i Let y be the probability that the model predicts pixel i as foreground. i ∈[0,1] is a pseudo-label; w H w M w B These are the loss weighting coefficients for the high-confidence prospect, medium-confidence prospect, and background regions, respectively, CE(p i ,y i () represents the classification cross-entropy loss function, used to calculate the prediction probability p of the segmentation model for a single pixel. i With pseudo-labels y i The differences between them.

8. The unsupervised segmentation method for concrete cracks based on CLIP large model and pseudo-mask self-purification as described in claim 7, characterized in that, Step S7 also includes the following: In step S7, the dynamic label update mechanism is executed on a fixed iteration cycle, with a fixed iteration interval divided into a training segment. At the end of each training segment, the model with the highest average recall rate on high-confidence foreground and background in the training set is saved as the optimal model for the current stage. The optimal model is used to update the current multi-level pseudomask. The update mechanism includes: for pixels in the current uncertain region that are predicted as background by the optimal model, when their background probability reaches a set threshold, they are updated as background by a certain proportion, gradually reducing the noise region in the initial pseudolabel; when the pixel change ratio in the uncertain region in this round is lower than a preset threshold, foreground denoising and foreground restoration operations are triggered. For pixels in the medium-confidence foreground region that are predicted as background by the optimal model and whose probability reaches the set threshold, they are downgraded to uncertain regions using a small batch method; at the same time, for pixels in the uncertain region of the initial pseudomask that are predicted as foreground by the optimal model and whose probability reaches the set threshold, they are marked as medium-confidence foreground regions, forming an update closed loop; the high-confidence foreground and background regions of the initial pseudomask do not participate in the update throughout the entire process to maintain the overall stability of the supervision signal.

9. An unsupervised concrete crack segmentation system based on CLIP large model and pseudo-mask self-purification, comprising an electronic device, wherein the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements an unsupervised segmentation method for concrete cracks based on CLIP large model and pseudo-mask self-purification as described in any one of claims 1 to 8.

10. An unsupervised concrete crack segmentation system based on CLIP large model and pseudo-mask self-purification, comprising a computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements an unsupervised segmentation method for concrete cracks based on CLIP large model and pseudo-mask self-purification as described in any one of claims 1 to 8.

Images